[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:26:27 dist-test-slave-9pj8 krb5kdc[10704](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:26:27 dist-test-slave-9pj8 krb5kdc[10704](info): set up 2 sockets
Sep 25 09:26:27 dist-test-slave-9pj8 krb5kdc[10704](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260925 09:26:28.146607 10697 mini_kdc.cc:121] Time spent starting KDC: real 2.764s user 0.007s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260925 09:26:28.298278 10697 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.151s user 0.000s sys 0.004s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260925 09:26:28.528661 10697 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.230s user 0.000s sys 0.004s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260925 09:26:28.695912 10697 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.167s user 0.000s sys 0.004s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/test-user.keytab.
W20260925 09:26:28.914894 10697 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.219s user 0.002s sys 0.002s
Sep 25 09:26:28 dist-test-slave-9pj8 krb5kdc[10704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:26:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:26:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:26:28.985016 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:42213
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:34433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:42213
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:29.286883 10717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:29.287129 10717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:29.287202 10717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:29.293623 10717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:26:29.293717 10717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:29.293759 10717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:29.293809 10717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:26:29.293849 10717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:26:29.303041 10717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:34433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:42213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:42213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:29.304176 10717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:29.305305 10717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:29.317018 10722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:29.317111 10723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:29.317314 10725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:29.318318 10717 server_base.cc:1034] running on GCE node
I20260925 09:26:29.319049 10717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:29.320364 10717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:29.321586 10717 hybrid_clock.cc:648] HybridClock initialized: now 1790328389321551 us; error 42 us; skew 500 ppm
I20260925 09:26:29.321915 10717 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:29 dist-test-slave-9pj8 krb5kdc[10704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328389, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:29.325311 10717 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:26:29.326798 10717 webserver.cc:460] Webserver started at http://127.10.114.126:41511/ using document root <none> and password file <none>
I20260925 09:26:29.327502 10717 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:29.327651 10717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:29.328033 10717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:29.331236 10717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "86de250a469448eb9b836737f3b66362"
format_stamp: "Formatted at 2026-09-25 09:26:29 on dist-test-slave-9pj8"
server_key: "24c7b4f78bb69430546cb1753e6e98ce"
server_key_iv: "f07b5f44be583626a732bcbd02f4a898"
server_key_version: "encryptionkey@0"
I20260925 09:26:29.332567 10717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "86de250a469448eb9b836737f3b66362"
format_stamp: "Formatted at 2026-09-25 09:26:29 on dist-test-slave-9pj8"
server_key: "24c7b4f78bb69430546cb1753e6e98ce"
server_key_iv: "f07b5f44be583626a732bcbd02f4a898"
server_key_version: "encryptionkey@0"
I20260925 09:26:29.337812 10717 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 09:26:29.341053 10732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:29.342278 10717 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:26:29.342476 10717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "86de250a469448eb9b836737f3b66362"
format_stamp: "Formatted at 2026-09-25 09:26:29 on dist-test-slave-9pj8"
server_key: "24c7b4f78bb69430546cb1753e6e98ce"
server_key_iv: "f07b5f44be583626a732bcbd02f4a898"
server_key_version: "encryptionkey@0"
I20260925 09:26:29.342669 10717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:29.377694 10717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:29.381896 10717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:29.382191 10717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:29.402498 10717 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:42213
I20260925 09:26:29.402499 10783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:42213 every 8 connection(s)
I20260925 09:26:29.403995 10717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:26:29.407611 10784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:29.408179 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 10717
I20260925 09:26:29.408473 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:26:29.410244 10697 external_mini_cluster.cc:1428] Setting key 0eed9edda19cbe1a7e469b5f1444b2e4
I20260925 09:26:29.418380 10784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362: Bootstrap starting.
I20260925 09:26:29.421216 10784 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:29.422109 10784 log.cc:824] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362: Log is configured to *not* fsync() on all Append() calls
I20260925 09:26:29.424714 10784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362: No bootstrap required, opened a new log
I20260925 09:26:29.431367 10784 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86de250a469448eb9b836737f3b66362" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42213 } }
I20260925 09:26:29.431728 10784 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:29.431861 10784 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86de250a469448eb9b836737f3b66362, State: Initialized, Role: FOLLOWER
I20260925 09:26:29.432516 10784 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86de250a469448eb9b836737f3b66362" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42213 } }
I20260925 09:26:29.432803 10784 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:26:29.432927 10784 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:26:29.433117 10784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:29.436300 10784 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86de250a469448eb9b836737f3b66362" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42213 } }
I20260925 09:26:29.436805 10784 leader_election.cc:302] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86de250a469448eb9b836737f3b66362; no voters:
Sep 25 09:26:29 dist-test-slave-9pj8 krb5kdc[10704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:29.437386 10784 leader_election.cc:288] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:26:29.437520 10789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:26:29.439855 10784 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:26:29.439890 10789 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362 [term 1 LEADER]: Becoming Leader. State: Replica: 86de250a469448eb9b836737f3b66362, State: Running, Role: LEADER
I20260925 09:26:29.440639 10789 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86de250a469448eb9b836737f3b66362" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42213 } }
I20260925 09:26:29.444095 10790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86de250a469448eb9b836737f3b66362" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86de250a469448eb9b836737f3b66362" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42213 } } }
I20260925 09:26:29.444550 10790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362 [sys.catalog]: This master's current role is: LEADER
I20260925 09:26:29.443737 10791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86de250a469448eb9b836737f3b66362. Latest consensus state: current_term: 1 leader_uuid: "86de250a469448eb9b836737f3b66362" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86de250a469448eb9b836737f3b66362" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42213 } } }
I20260925 09:26:29.445317 10791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362 [sys.catalog]: This master's current role is: LEADER
I20260925 09:26:29.453377 10786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:29.413764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43138 (local address 127.10.114.126:42213)
0925 09:26:29.414484 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:29.414494 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:29.415684 (+ 1190us) server_negotiation.cc:407] Connection header received
0925 09:26:29.424396 (+ 8712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:29.424415 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:29.424731 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:29.425115 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:29.428173 (+ 3058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:29.429462 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:29.431015 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:29.431622 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:29.431858 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:29.439510 (+ 7652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:29.439520 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:29.439548 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:29.439597 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:29.444795 (+ 5198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:29.446033 (+ 1238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:29.446044 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:29.446059 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:29.446314 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:29.446951 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:29.446959 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:29.446964 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:29.447721 (+ 757us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:29.451025 (+ 3304us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:29.451326 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 09:26:29.451863 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":351,"threads_started":1}
I20260925 09:26:29.460350 10795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:26:29.466813 10795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:26:29.475253 10795 catalog_manager.cc:1347] Generated new cluster ID: da4124fe7b6d40ec967245a70c98cb1b
I20260925 09:26:29.475344 10795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:26:29.496760 10795 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:26:29.498648 10795 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:26:29.508903 10795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86de250a469448eb9b836737f3b66362: Generated new TSK 0
I20260925 09:26:29.509912 10795 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
I20260925 09:26:29.553018 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42213
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:34433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:29.864554 10811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:29.864796 10811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:29.864893 10811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:29.871569 10811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:29.871661 10811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:29.871765 10811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:26:29.881244 10811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:34433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42213
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:29.882347 10811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:29.883548 10811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:29.896490 10819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:29.896621 10817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:29.896834 10816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:29.898029 10811 server_base.cc:1034] running on GCE node
I20260925 09:26:29.898636 10811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:29.899498 10811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:29.900683 10811 hybrid_clock.cc:648] HybridClock initialized: now 1790328389900624 us; error 65 us; skew 500 ppm
I20260925 09:26:29.900988 10811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:29 dist-test-slave-9pj8 krb5kdc[10704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328389, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:29.904062 10811 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:26:29.905594 10811 webserver.cc:460] Webserver started at http://127.10.114.65:38371/ using document root <none> and password file <none>
I20260925 09:26:29.906241 10811 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:29.906338 10811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:29.906664 10811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:29.909432 10811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "b15a69fdeb1b4eb3876e854ed0b10f02"
format_stamp: "Formatted at 2026-09-25 09:26:29 on dist-test-slave-9pj8"
server_key: "9a62b2102251f3a5d1ff9319f827d5de"
server_key_iv: "1000f0c30d57b1fad20bb626cf8daa17"
server_key_version: "encryptionkey@0"
I20260925 09:26:29.910180 10811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "b15a69fdeb1b4eb3876e854ed0b10f02"
format_stamp: "Formatted at 2026-09-25 09:26:29 on dist-test-slave-9pj8"
server_key: "9a62b2102251f3a5d1ff9319f827d5de"
server_key_iv: "1000f0c30d57b1fad20bb626cf8daa17"
server_key_version: "encryptionkey@0"
I20260925 09:26:29.915343 10811 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260925 09:26:29.918645 10826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:29.920063 10811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:26:29.920217 10811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "b15a69fdeb1b4eb3876e854ed0b10f02"
format_stamp: "Formatted at 2026-09-25 09:26:29 on dist-test-slave-9pj8"
server_key: "9a62b2102251f3a5d1ff9319f827d5de"
server_key_iv: "1000f0c30d57b1fad20bb626cf8daa17"
server_key_version: "encryptionkey@0"
I20260925 09:26:29.920395 10811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:29.941293 10811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:29.944952 10811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:29.945202 10811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:29.946101 10811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:26:29.947861 10811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:26:29.947937 10811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.001s
I20260925 09:26:29.948050 10811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:26:29.948112 10811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:29.981171 10811 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:33255
I20260925 09:26:29.981249 10938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:33255 every 8 connection(s)
I20260925 09:26:29.982409 10811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:26:29.986445 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 10811
I20260925 09:26:29.986663 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:26:29.987453 10697 external_mini_cluster.cc:1428] Setting key b048983a087bd98ffbd5b933d20dfff4
Sep 25 09:26:29 dist-test-slave-9pj8 krb5kdc[10704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328389, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.114.66@KRBTEST.COM; defaulting to no policy
I20260925 09:26:30.004791 10941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:29.984844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:52121 (local address 127.10.114.126:42213)
0925 09:26:29.985219 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:29.985224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:29.986382 (+ 1158us) server_negotiation.cc:407] Connection header received
0925 09:26:29.988685 (+ 2303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:29.988690 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:29.988771 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:29.988885 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:29.990616 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:29.991297 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:29.993529 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:29.993982 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:29.994107 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:29.997933 (+ 3826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:29.997941 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:29.997948 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:29.997990 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:30.001588 (+ 3598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:30.002371 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:30.002377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:30.002380 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:30.002543 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:30.003017 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:30.003025 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:30.003030 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:30.003407 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:30.003563 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:30.004255 (+ 692us) server_negotiation.cc:299] Negotiation successful
0925 09:26:30.004511 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":189,"threads_started":1}
I20260925 09:26:30.005640 10942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:29.985394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42213 (local address 127.10.114.65:52121)
0925 09:26:29.986130 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:29.986194 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:29.988448 (+ 2254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:29.989059 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:29.989066 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:29.989657 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:29.990427 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:29.990458 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:29.991449 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:29.991457 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:29.993411 (+ 1954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:29.993426 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:29.994165 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:29.994169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:29.994346 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:29.995111 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:29.995140 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:29.997709 (+ 2569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:30.001763 (+ 4054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:30.001770 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:30.001782 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:30.002193 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:30.002700 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:30.002704 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:30.002706 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:30.002889 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:30.003551 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:30.003556 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:30.003915 (+ 359us) client_negotiation.cc:769] Sending connection context
0925 09:26:30.004119 (+ 204us) client_negotiation.cc:240] Negotiation successful
0925 09:26:30.004522 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":260,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.66@KRBTEST.COM" created.
I20260925 09:26:30.007328 10939 heartbeater.cc:347] Connected to a master server at 127.10.114.126:42213
I20260925 09:26:30.007639 10939 heartbeater.cc:464] Registering TS with master...
I20260925 09:26:30.008440 10939 heartbeater.cc:511] Master 127.10.114.126:42213 requested a full tablet report, sending...
I20260925 09:26:30.010627 10749 ts_manager.cc:194] Registered new tserver with Master: b15a69fdeb1b4eb3876e854ed0b10f02 (127.10.114.65:33255)
I20260925 09:26:30.012158 10749 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:52121
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
I20260925 09:26:30.027101 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42213
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:34433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:30.349620 10945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:30.349970 10945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:30.350149 10945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:30.360183 10945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:30.360309 10945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:30.360507 10945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:26:30.369298 10945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:34433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42213
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:30.370476 10945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:30.371754 10945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:30.384212 10950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:30.384112 10951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:30.384150 10953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:30.385140 10945 server_base.cc:1034] running on GCE node
I20260925 09:26:30.385632 10945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:30.386539 10945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:30.387759 10945 hybrid_clock.cc:648] HybridClock initialized: now 1790328390387709 us; error 43 us; skew 500 ppm
I20260925 09:26:30.388070 10945 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:30 dist-test-slave-9pj8 krb5kdc[10704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328390, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:30.390765 10945 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:26:30.392177 10945 webserver.cc:460] Webserver started at http://127.10.114.66:36865/ using document root <none> and password file <none>
I20260925 09:26:30.392802 10945 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:30.392947 10945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:30.393232 10945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:30.395995 10945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "21c40017dc3d46edb6a37641ceaf47d4"
format_stamp: "Formatted at 2026-09-25 09:26:30 on dist-test-slave-9pj8"
server_key: "5edc8b8f3cc96b2cc267d81835128ab7"
server_key_iv: "5d861632f6e076f30217984f1287e0a5"
server_key_version: "encryptionkey@0"
I20260925 09:26:30.396785 10945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "21c40017dc3d46edb6a37641ceaf47d4"
format_stamp: "Formatted at 2026-09-25 09:26:30 on dist-test-slave-9pj8"
server_key: "5edc8b8f3cc96b2cc267d81835128ab7"
server_key_iv: "5d861632f6e076f30217984f1287e0a5"
server_key_version: "encryptionkey@0"
I20260925 09:26:30.401942 10945 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 09:26:30.405401 10960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:30.406881 10945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260925 09:26:30.407023 10945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "21c40017dc3d46edb6a37641ceaf47d4"
format_stamp: "Formatted at 2026-09-25 09:26:30 on dist-test-slave-9pj8"
server_key: "5edc8b8f3cc96b2cc267d81835128ab7"
server_key_iv: "5d861632f6e076f30217984f1287e0a5"
server_key_version: "encryptionkey@0"
I20260925 09:26:30.407203 10945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:30.428689 10945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:30.432574 10945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:30.432770 10945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:30.433593 10945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:26:30.435127 10945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:26:30.435204 10945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:30.435277 10945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:26:30.435325 10945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:30.466611 10945 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:33347
I20260925 09:26:30.466650 11072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:33347 every 8 connection(s)
I20260925 09:26:30.467832 10945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:26:30.471292 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 10945
I20260925 09:26:30.471468 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:26:30.471963 10697 external_mini_cluster.cc:1428] Setting key 74f6a1a516e34106e84df2321f38a09d
Sep 25 09:26:30 dist-test-slave-9pj8 krb5kdc[10704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328390, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.114.67@KRBTEST.COM; defaulting to no policy
I20260925 09:26:30.488134 10941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:30.470180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:54679 (local address 127.10.114.126:42213)
0925 09:26:30.470352 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:30.470357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:30.471456 (+ 1099us) server_negotiation.cc:407] Connection header received
0925 09:26:30.473443 (+ 1987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:30.473448 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:30.473549 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:30.473652 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:30.474993 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:30.475658 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:30.477949 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:30.478327 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:30.478419 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:30.482247 (+ 3828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:30.482251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:30.482255 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:30.482281 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:30.484872 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:30.485613 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:30.485625 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:30.485628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:30.485800 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:30.486292 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:30.486304 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:30.486306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:30.486566 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:30.486892 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:30.487533 (+ 641us) server_negotiation.cc:299] Negotiation successful
0925 09:26:30.487806 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:26:30.489147 11075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:30.470636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42213 (local address 127.10.114.66:54679)
0925 09:26:30.471296 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:30.471331 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:30.473191 (+ 1860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:30.473872 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:30.473879 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:30.474333 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:30.474833 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:30.474849 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:30.475848 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:30.475858 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:30.477787 (+ 1929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:30.477805 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:30.478538 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:30.478547 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:30.478849 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:30.479543 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:30.479580 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:30.482047 (+ 2467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:30.485021 (+ 2974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:30.485030 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:30.485045 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:30.485475 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:30.485981 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:30.485986 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:30.485990 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:30.486172 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:30.486880 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:30.486888 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:30.487178 (+ 290us) client_negotiation.cc:769] Sending connection context
0925 09:26:30.487437 (+ 259us) client_negotiation.cc:240] Negotiation successful
0925 09:26:30.487953 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":274,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.67@KRBTEST.COM" created.
I20260925 09:26:30.490532 11073 heartbeater.cc:347] Connected to a master server at 127.10.114.126:42213
I20260925 09:26:30.490828 11073 heartbeater.cc:464] Registering TS with master...
I20260925 09:26:30.491654 11073 heartbeater.cc:511] Master 127.10.114.126:42213 requested a full tablet report, sending...
I20260925 09:26:30.493093 10749 ts_manager.cc:194] Registered new tserver with Master: 21c40017dc3d46edb6a37641ceaf47d4 (127.10.114.66:33347)
I20260925 09:26:30.493840 10749 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:54679
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
I20260925 09:26:30.511029 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42213
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:34433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:30.819068 11078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:30.819329 11078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:30.819482 11078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:30.827502 11078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:30.827589 11078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:30.827744 11078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:26:30.836691 11078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:34433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42213
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:30.837898 11078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:30.839195 11078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:30.852185 11083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:30.852264 11084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:30.852289 11086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:30.853020 11078 server_base.cc:1034] running on GCE node
I20260925 09:26:30.853557 11078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:30.854417 11078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:30.855620 11078 hybrid_clock.cc:648] HybridClock initialized: now 1790328390855569 us; error 51 us; skew 500 ppm
I20260925 09:26:30.856045 11078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:30 dist-test-slave-9pj8 krb5kdc[10704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328390, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:30.858954 11078 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:26:30.860268 11078 webserver.cc:460] Webserver started at http://127.10.114.67:45087/ using document root <none> and password file <none>
I20260925 09:26:30.860949 11078 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:30.861086 11078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:30.861364 11078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:30.864507 11078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "843b63859be2428782beb557cfb4494a"
format_stamp: "Formatted at 2026-09-25 09:26:30 on dist-test-slave-9pj8"
server_key: "75bc89d41b7880d7949569943b95bbfa"
server_key_iv: "f7e45c43640e11a0229bc57bbf467291"
server_key_version: "encryptionkey@0"
I20260925 09:26:30.865149 11078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "843b63859be2428782beb557cfb4494a"
format_stamp: "Formatted at 2026-09-25 09:26:30 on dist-test-slave-9pj8"
server_key: "75bc89d41b7880d7949569943b95bbfa"
server_key_iv: "f7e45c43640e11a0229bc57bbf467291"
server_key_version: "encryptionkey@0"
I20260925 09:26:30.869565 11078 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:26:30.872645 11093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:30.873801 11078 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 09:26:30.874040 11078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "843b63859be2428782beb557cfb4494a"
format_stamp: "Formatted at 2026-09-25 09:26:30 on dist-test-slave-9pj8"
server_key: "75bc89d41b7880d7949569943b95bbfa"
server_key_iv: "f7e45c43640e11a0229bc57bbf467291"
server_key_version: "encryptionkey@0"
I20260925 09:26:30.874212 11078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:30.902338 11078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:30.906442 11078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:30.906672 11078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:30.907487 11078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:26:30.908668 11078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:26:30.908735 11078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:30.908807 11078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:26:30.908854 11078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:30.941331 11078 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:45401
I20260925 09:26:30.941418 11205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:45401 every 8 connection(s)
I20260925 09:26:30.942601 11078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:26:30.945087 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 11078
I20260925 09:26:30.945286 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:26:30.945696 10697 external_mini_cluster.cc:1428] Setting key 5f96a3fe3152aafdbebf43be11bf91d0
Sep 25 09:26:30 dist-test-slave-9pj8 krb5kdc[10704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328390, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:30.960937 10941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:30.945252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:44487 (local address 127.10.114.126:42213)
0925 09:26:30.945443 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:30.945449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:30.946738 (+ 1289us) server_negotiation.cc:407] Connection header received
0925 09:26:30.948863 (+ 2125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:30.948868 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:30.948944 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:30.949020 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:30.950368 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:30.951042 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:30.952451 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:30.952816 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:30.952900 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:30.955956 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:30.955960 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:30.955963 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:30.955988 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:30.958146 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:30.958724 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:30.958727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:30.958729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:30.958835 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:30.959173 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:30.959176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:30.959177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:30.959440 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:30.959584 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:30.960430 (+ 846us) server_negotiation.cc:299] Negotiation successful
0925 09:26:30.960599 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 09:26:30.961841 11208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:30.945895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42213 (local address 127.10.114.67:44487)
0925 09:26:30.946582 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:30.946643 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:30.948628 (+ 1985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:30.949271 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:30.949278 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:30.949739 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:30.950236 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:30.950252 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:30.951168 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:30.951171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:30.952335 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:30.952345 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:30.952971 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:30.952974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:30.953199 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:30.953806 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:30.953825 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:30.955736 (+ 1911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:30.958254 (+ 2518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:30.958260 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:30.958270 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:30.958626 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:30.958925 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:30.958928 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:30.958930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:30.959085 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:30.959582 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:30.959590 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:30.960017 (+ 427us) client_negotiation.cc:769] Sending connection context
0925 09:26:30.960311 (+ 294us) client_negotiation.cc:240] Negotiation successful
0925 09:26:30.960727 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":297,"threads_started":1}
I20260925 09:26:30.963057 11206 heartbeater.cc:347] Connected to a master server at 127.10.114.126:42213
I20260925 09:26:30.963289 11206 heartbeater.cc:464] Registering TS with master...
I20260925 09:26:30.963934 11206 heartbeater.cc:511] Master 127.10.114.126:42213 requested a full tablet report, sending...
I20260925 09:26:30.965093 10749 ts_manager.cc:194] Registered new tserver with Master: 843b63859be2428782beb557cfb4494a (127.10.114.67:45401)
I20260925 09:26:30.965787 10749 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:44487
I20260925 09:26:30.973189 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 09:26:30 dist-test-slave-9pj8 krb5kdc[10704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328390, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 09:26:31 dist-test-slave-9pj8 krb5kdc[10704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328390, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:26:31.014897 10939 heartbeater.cc:503] Master 127.10.114.126:42213 was elected leader, sending a full tablet report...
I20260925 09:26:31.018745 11213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:31.002840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49878 (local address 127.10.114.65:33255)
0925 09:26:31.003358 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:31.003365 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:31.003534 (+ 169us) server_negotiation.cc:407] Connection header received
0925 09:26:31.003706 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:31.003712 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:31.003979 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:31.004093 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:31.005662 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:31.006639 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:31.008098 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:31.008625 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:31.008724 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:31.012796 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:31.012806 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:31.012820 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:31.012852 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:31.015901 (+ 3049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:31.016789 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:31.016801 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:31.016804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:31.016948 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:31.017436 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:31.017439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:31.017441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:31.017764 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:31.017984 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:31.018286 (+ 302us) server_negotiation.cc:299] Negotiation successful
0925 09:26:31.018469 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":278,"threads_started":1}
W20260925 09:26:31.020061 10854 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49878
Sep 25 09:26:31 dist-test-slave-9pj8 krb5kdc[10704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 09:26:31 dist-test-slave-9pj8 krb5kdc[10704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:26:31.061700 11213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:31.048578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49888 (local address 127.10.114.65:33255)
0925 09:26:31.048788 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:31.048794 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:31.049006 (+ 212us) server_negotiation.cc:407] Connection header received
0925 09:26:31.049199 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:31.049211 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:31.049323 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:31.049432 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:31.050676 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:31.051375 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:31.052604 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:31.053001 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:31.053140 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:31.056929 (+ 3789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:31.056934 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:31.056937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:31.056965 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:31.059300 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:31.059896 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:31.059903 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:31.059906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:31.060021 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:31.060381 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:31.060385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:31.060390 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:31.060633 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:31.060861 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:31.061163 (+ 302us) server_negotiation.cc:299] Negotiation successful
0925 09:26:31.061399 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 09:26:31.063225 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 10811
I20260925 09:26:31.075696 11213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:31.064121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49896 (local address 127.10.114.65:33255)
0925 09:26:31.064284 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:31.064288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:31.064664 (+ 376us) server_negotiation.cc:407] Connection header received
0925 09:26:31.064736 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:31.064739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:31.064793 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:31.064887 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:31.066231 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:31.066924 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:31.068628 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:31.069049 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:31.069150 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:31.071221 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:31.071225 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:31.071229 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:31.071256 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:31.073691 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:31.074213 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:31.074216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:31.074218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:31.074331 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:31.074669 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:31.074672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:31.074675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:31.074966 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:31.075133 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:31.075314 (+ 181us) server_negotiation.cc:299] Negotiation successful
0925 09:26:31.075450 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 09:26:31.076328 10934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:26:31.157076 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 10811
I20260925 09:26:31.174118 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 10945
Sep 25 09:26:31 dist-test-slave-9pj8 krb5kdc[10704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.66@KRBTEST.COM
I20260925 09:26:31.192099 11220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:31.175051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49876 (local address 127.10.114.66:33347)
0925 09:26:31.175566 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:31.175573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:31.175600 (+ 27us) server_negotiation.cc:407] Connection header received
0925 09:26:31.175671 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:31.175689 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:31.176023 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:31.176164 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:31.177234 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:31.178355 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:31.179652 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:31.180362 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:31.180456 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:31.184167 (+ 3711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:31.184177 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:31.184190 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:31.184218 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:31.189775 (+ 5557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:31.190309 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:31.190315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:31.190318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:31.190434 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:31.190798 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:31.190802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:31.190804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:31.191141 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:31.191365 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:31.191595 (+ 230us) server_negotiation.cc:299] Negotiation successful
0925 09:26:31.191766 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":261,"threads_started":1}
I20260925 09:26:31.193284 11068 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:26:31.270622 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 10945
I20260925 09:26:31.292754 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 11078
Sep 25 09:26:31 dist-test-slave-9pj8 krb5kdc[10704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.67@KRBTEST.COM
I20260925 09:26:31.308405 11222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:31.293807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57088 (local address 127.10.114.67:45401)
0925 09:26:31.294401 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:31.294420 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:31.294446 (+ 26us) server_negotiation.cc:407] Connection header received
0925 09:26:31.294500 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:31.294506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:31.294699 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:31.294817 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:31.296036 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:31.296981 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:31.298216 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:31.298757 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:31.298868 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:31.302995 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:31.303007 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:31.303020 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:31.303051 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:31.305875 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:31.306489 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:31.306496 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:31.306499 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:31.306620 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:31.306961 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:31.306964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:31.306966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:31.307286 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:31.307530 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:31.307924 (+ 394us) server_negotiation.cc:299] Negotiation successful
0925 09:26:31.308126 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":309,"threads_started":1}
I20260925 09:26:31.309707 11201 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:26:31.387677 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 11078
I20260925 09:26:31.403632 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 10717
I20260925 09:26:31.404783 10779 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:26:31.500684 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 10717
2026-09-25T09:26:31Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (6167 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:26:31 dist-test-slave-9pj8 krb5kdc[11227](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:26:31 dist-test-slave-9pj8 krb5kdc[11227](info): set up 2 sockets
Sep 25 09:26:31 dist-test-slave-9pj8 krb5kdc[11227](info): commencing operation
krb5kdc: starting...
W20260925 09:26:32.544350 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.005s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:26:32 dist-test-slave-9pj8 krb5kdc[11227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:26:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:26:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:26:32.707882 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:39305
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:39655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:39305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:33.014648 11240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:33.014999 11240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:33.015187 11240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:33.022415 11240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:26:33.022578 11240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:33.022694 11240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:33.022800 11240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:26:33.022889 11240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:26:33.032028 11240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:39655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:39305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:39305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:33.033219 11240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:33.034477 11240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:33.045573 11245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:33.045621 11246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:33.045967 11248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:33.047060 11240 server_base.cc:1034] running on GCE node
I20260925 09:26:33.047991 11240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:33.049876 11240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:33.051196 11240 hybrid_clock.cc:648] HybridClock initialized: now 1790328393051070 us; error 101 us; skew 500 ppm
I20260925 09:26:33.051692 11240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:33 dist-test-slave-9pj8 krb5kdc[11227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328393, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:33.054929 11240 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:26:33.056255 11240 webserver.cc:460] Webserver started at http://127.10.114.126:37443/ using document root <none> and password file <none>
I20260925 09:26:33.056906 11240 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:33.057034 11240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:33.057343 11240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:33.060178 11240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "455d13df169f41478cbbf380366fa896"
format_stamp: "Formatted at 2026-09-25 09:26:33 on dist-test-slave-9pj8"
server_key: "d6cdb0a144184bdbda5ef4f673edf91f"
server_key_iv: "7479ae0ff908ef9d602e4f0c43284f93"
server_key_version: "encryptionkey@0"
I20260925 09:26:33.060838 11240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "455d13df169f41478cbbf380366fa896"
format_stamp: "Formatted at 2026-09-25 09:26:33 on dist-test-slave-9pj8"
server_key: "d6cdb0a144184bdbda5ef4f673edf91f"
server_key_iv: "7479ae0ff908ef9d602e4f0c43284f93"
server_key_version: "encryptionkey@0"
I20260925 09:26:33.065353 11240 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:26:33.068444 11255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:33.069607 11240 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 09:26:33.069753 11240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "455d13df169f41478cbbf380366fa896"
format_stamp: "Formatted at 2026-09-25 09:26:33 on dist-test-slave-9pj8"
server_key: "d6cdb0a144184bdbda5ef4f673edf91f"
server_key_iv: "7479ae0ff908ef9d602e4f0c43284f93"
server_key_version: "encryptionkey@0"
I20260925 09:26:33.069949 11240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:33.112978 11240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:33.116709 11240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:33.116971 11240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:33.134486 11240 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:39305
I20260925 09:26:33.134531 11306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:39305 every 8 connection(s)
I20260925 09:26:33.135675 11240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:26:33.139329 11307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:33.141342 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 11240
I20260925 09:26:33.141585 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:26:33.142452 10697 external_mini_cluster.cc:1428] Setting key fce79a8b6e3261f1f074dedc59c7d335
I20260925 09:26:33.153373 11307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896: Bootstrap starting.
Sep 25 09:26:33 dist-test-slave-9pj8 krb5kdc[11227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:33.156921 11307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:33.158042 11307 log.cc:824] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896: Log is configured to *not* fsync() on all Append() calls
I20260925 09:26:33.160741 11307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896: No bootstrap required, opened a new log
I20260925 09:26:33.164667 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:33.144407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48098 (local address 127.10.114.126:39305)
0925 09:26:33.145099 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:33.145113 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:33.145154 (+ 41us) server_negotiation.cc:407] Connection header received
0925 09:26:33.146590 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:33.146610 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:33.146979 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:33.147292 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:33.149040 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:33.150344 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:33.152070 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:33.152659 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:33.152777 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:33.157191 (+ 4414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:33.157198 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:33.157212 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:33.157245 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:33.160322 (+ 3077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:33.160975 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:33.161002 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:33.161009 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:33.161228 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:33.161588 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:33.161594 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:33.161598 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:33.162247 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:33.162534 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:33.162904 (+ 370us) server_negotiation.cc:299] Negotiation successful
0925 09:26:33.163355 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":316,"threads_started":1}
I20260925 09:26:33.169046 11307 raft_consensus.cc:348] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455d13df169f41478cbbf380366fa896" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39305 } }
I20260925 09:26:33.169312 11307 raft_consensus.cc:374] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:33.169376 11307 raft_consensus.cc:729] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 455d13df169f41478cbbf380366fa896, State: Initialized, Role: FOLLOWER
I20260925 09:26:33.169845 11307 consensus_queue.cc:260] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455d13df169f41478cbbf380366fa896" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39305 } }
I20260925 09:26:33.170027 11307 raft_consensus.cc:388] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:26:33.170100 11307 raft_consensus.cc:482] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:26:33.170208 11307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:33.172119 11307 raft_consensus.cc:504] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455d13df169f41478cbbf380366fa896" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39305 } }
I20260925 09:26:33.172501 11307 leader_election.cc:302] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 455d13df169f41478cbbf380366fa896; no voters:
I20260925 09:26:33.173069 11307 leader_election.cc:288] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:26:33.173225 11312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:26:33.174288 11312 raft_consensus.cc:686] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896 [term 1 LEADER]: Becoming Leader. State: Replica: 455d13df169f41478cbbf380366fa896, State: Running, Role: LEADER
I20260925 09:26:33.174813 11312 consensus_queue.cc:237] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455d13df169f41478cbbf380366fa896" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39305 } }
I20260925 09:26:33.175032 11307 sys_catalog.cc:564] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:26:33.177374 11313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "455d13df169f41478cbbf380366fa896" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455d13df169f41478cbbf380366fa896" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39305 } } }
I20260925 09:26:33.177727 11313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896 [sys.catalog]: This master's current role is: LEADER
I20260925 09:26:33.178730 11314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 455d13df169f41478cbbf380366fa896. Latest consensus state: current_term: 1 leader_uuid: "455d13df169f41478cbbf380366fa896" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455d13df169f41478cbbf380366fa896" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39305 } } }
I20260925 09:26:33.179083 11314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896 [sys.catalog]: This master's current role is: LEADER
I20260925 09:26:33.181506 11319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:26:33.186600 11319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:26:33.194264 11319 catalog_manager.cc:1347] Generated new cluster ID: e032030e52454f499289d9a9d5657fb9
I20260925 09:26:33.194355 11319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:26:33.225831 11319 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:26:33.227065 11319 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:26:33.236002 11319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 455d13df169f41478cbbf380366fa896: Generated new TSK 0
I20260925 09:26:33.236826 11319 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
I20260925 09:26:33.283690 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39305
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:39655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:33.580205 11333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:33.580448 11333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:33.580611 11333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:33.587424 11333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:33.587517 11333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:33.587669 11333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:26:33.596669 11333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:39655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39305
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:33.597853 11333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:33.599166 11333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:33.611626 11338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:33.611603 11341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:33.611836 11339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:33.613042 11333 server_base.cc:1034] running on GCE node
I20260925 09:26:33.613570 11333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:33.614408 11333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:33.615628 11333 hybrid_clock.cc:648] HybridClock initialized: now 1790328393615576 us; error 50 us; skew 500 ppm
I20260925 09:26:33.615971 11333 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:33 dist-test-slave-9pj8 krb5kdc[11227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328393, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:33.619270 11333 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:26:33.620767 11333 webserver.cc:460] Webserver started at http://127.10.114.65:38809/ using document root <none> and password file <none>
I20260925 09:26:33.621476 11333 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:33.621624 11333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:33.621945 11333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:33.624903 11333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "8a32dfaf4cfb44c684bb7d082e45f4f3"
format_stamp: "Formatted at 2026-09-25 09:26:33 on dist-test-slave-9pj8"
server_key: "2a1b923bc29f46abd5fdb7638771c119"
server_key_iv: "259fbbf2486d409889a332ca2ff2936b"
server_key_version: "encryptionkey@0"
I20260925 09:26:33.625703 11333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "8a32dfaf4cfb44c684bb7d082e45f4f3"
format_stamp: "Formatted at 2026-09-25 09:26:33 on dist-test-slave-9pj8"
server_key: "2a1b923bc29f46abd5fdb7638771c119"
server_key_iv: "259fbbf2486d409889a332ca2ff2936b"
server_key_version: "encryptionkey@0"
I20260925 09:26:33.630710 11333 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:26:33.634342 11348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:33.636085 11333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260925 09:26:33.636240 11333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "8a32dfaf4cfb44c684bb7d082e45f4f3"
format_stamp: "Formatted at 2026-09-25 09:26:33 on dist-test-slave-9pj8"
server_key: "2a1b923bc29f46abd5fdb7638771c119"
server_key_iv: "259fbbf2486d409889a332ca2ff2936b"
server_key_version: "encryptionkey@0"
I20260925 09:26:33.636536 11333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:33.676875 11333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:33.681056 11333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:33.681407 11333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:33.682418 11333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:26:33.683833 11333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:26:33.683941 11333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:33.684096 11333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:26:33.684170 11333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:33.717434 11333 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:42979
I20260925 09:26:33.717471 11460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:42979 every 8 connection(s)
I20260925 09:26:33.718830 11333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:26:33.728148 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 11333
I20260925 09:26:33.728348 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:26:33.728961 10697 external_mini_cluster.cc:1428] Setting key 0031b811e8b56c81ffd79d49ad5beb33
Sep 25 09:26:33 dist-test-slave-9pj8 krb5kdc[11227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328393, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:33.738387 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:33.721632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:40807 (local address 127.10.114.126:39305)
0925 09:26:33.722081 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:33.722087 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:33.723062 (+ 975us) server_negotiation.cc:407] Connection header received
0925 09:26:33.724927 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:33.724931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:33.724993 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:33.725093 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:33.726501 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:33.727241 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:33.728912 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:33.729493 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:33.729658 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:33.733018 (+ 3360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:33.733022 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:33.733025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:33.733056 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:33.735471 (+ 2415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:33.736227 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:33.736231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:33.736233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:33.736337 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:33.736750 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:33.736755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:33.736758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:33.737004 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:33.737174 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:33.737818 (+ 644us) server_negotiation.cc:299] Negotiation successful
0925 09:26:33.738051 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":216,"threads_started":1}
I20260925 09:26:33.739449 11464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:33.722202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39305 (local address 127.10.114.65:40807)
0925 09:26:33.722892 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:33.722936 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:33.724675 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:33.725294 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:33.725302 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:33.725774 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:33.726328 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:33.726345 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:33.727376 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:33.727380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:33.728772 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:33.728784 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:33.729732 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:33.729739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:33.729974 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:33.730722 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:33.730753 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:33.732813 (+ 2060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:33.735611 (+ 2798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:33.735620 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:33.735634 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:33.736089 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:33.736451 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:33.736457 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:33.736460 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:33.736638 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:33.737159 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:33.737166 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:33.737468 (+ 302us) client_negotiation.cc:769] Sending connection context
0925 09:26:33.737721 (+ 253us) client_negotiation.cc:240] Negotiation successful
0925 09:26:33.738199 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":268,"threads_started":1}
I20260925 09:26:33.741065 11461 heartbeater.cc:347] Connected to a master server at 127.10.114.126:39305
I20260925 09:26:33.741351 11461 heartbeater.cc:464] Registering TS with master...
I20260925 09:26:33.742023 11461 heartbeater.cc:511] Master 127.10.114.126:39305 requested a full tablet report, sending...
I20260925 09:26:33.744035 11272 ts_manager.cc:194] Registered new tserver with Master: 8a32dfaf4cfb44c684bb7d082e45f4f3 (127.10.114.65:42979)
WARNING: no policy specified for kudu/127.10.114.66@KRBTEST.COM; defaulting to no policy
I20260925 09:26:33.745404 11272 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:40807
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
I20260925 09:26:33.767444 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39305
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:39655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:34.071393 11467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:34.071623 11467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:34.071754 11467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:34.078588 11467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:34.078677 11467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:34.078897 11467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:26:34.088200 11467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:39655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39305
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:34.089313 11467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:34.090667 11467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:34.102798 11473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:34.102768 11475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:34.102826 11472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:34.103647 11467 server_base.cc:1034] running on GCE node
I20260925 09:26:34.104226 11467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:34.105044 11467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:34.106235 11467 hybrid_clock.cc:648] HybridClock initialized: now 1790328394106187 us; error 48 us; skew 500 ppm
I20260925 09:26:34.106523 11467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:34 dist-test-slave-9pj8 krb5kdc[11227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328394, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:34.109766 11467 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:26:34.111173 11467 webserver.cc:460] Webserver started at http://127.10.114.66:45431/ using document root <none> and password file <none>
I20260925 09:26:34.111855 11467 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:34.111979 11467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:34.112304 11467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:34.114873 11467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "0207ee4d84514b2e843a30e65f15ff9f"
format_stamp: "Formatted at 2026-09-25 09:26:34 on dist-test-slave-9pj8"
server_key: "801fae0e8018396c54ea8c37eba2ffce"
server_key_iv: "864aa310f684df6096d95a12ac720432"
server_key_version: "encryptionkey@0"
I20260925 09:26:34.115600 11467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "0207ee4d84514b2e843a30e65f15ff9f"
format_stamp: "Formatted at 2026-09-25 09:26:34 on dist-test-slave-9pj8"
server_key: "801fae0e8018396c54ea8c37eba2ffce"
server_key_iv: "864aa310f684df6096d95a12ac720432"
server_key_version: "encryptionkey@0"
I20260925 09:26:34.120759 11467 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260925 09:26:34.124114 11482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:34.125335 11467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260925 09:26:34.125499 11467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "0207ee4d84514b2e843a30e65f15ff9f"
format_stamp: "Formatted at 2026-09-25 09:26:34 on dist-test-slave-9pj8"
server_key: "801fae0e8018396c54ea8c37eba2ffce"
server_key_iv: "864aa310f684df6096d95a12ac720432"
server_key_version: "encryptionkey@0"
I20260925 09:26:34.125701 11467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:34.148950 11467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:34.154042 11467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:34.154350 11467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:34.155473 11467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:26:34.156924 11467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:26:34.156999 11467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:34.157095 11467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:26:34.157145 11467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:34.189133 11467 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:45487
I20260925 09:26:34.189173 11594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:45487 every 8 connection(s)
I20260925 09:26:34.190321 11467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:26:34.191922 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 11467
I20260925 09:26:34.192104 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:26:34.192536 10697 external_mini_cluster.cc:1428] Setting key aa358424aa3213467ec0a61dc188d5e4
Sep 25 09:26:34 dist-test-slave-9pj8 krb5kdc[11227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328394, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.114.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.67@KRBTEST.COM" created.
I20260925 09:26:34.212064 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:34.193016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:34875 (local address 127.10.114.126:39305)
0925 09:26:34.193215 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:34.193221 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:34.194735 (+ 1514us) server_negotiation.cc:407] Connection header received
0925 09:26:34.196606 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:34.196614 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:34.196702 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:34.196828 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:34.198283 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.198994 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.200622 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.201403 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.201541 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:34.205334 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:34.205342 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:34.205350 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:34.205413 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:34.209087 (+ 3674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:34.209736 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:34.209741 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:34.209743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:34.209910 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:34.210300 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:34.210303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:34.210305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:34.210589 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:34.210727 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:34.211453 (+ 726us) server_negotiation.cc:299] Negotiation successful
0925 09:26:34.211735 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 09:26:34.213223 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:34.193614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39305 (local address 127.10.114.66:34875)
0925 09:26:34.194567 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:34.194618 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:34.196345 (+ 1727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:34.197014 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:34.197021 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:34.197507 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:34.198088 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:34.198116 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.199162 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.199166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:34.200469 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:34.200480 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.201601 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.201606 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:34.201784 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:34.202462 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:34.202490 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:34.205051 (+ 2561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:34.209218 (+ 4167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:34.209225 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:34.209237 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:34.209618 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:34.210006 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:34.210009 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:34.210011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:34.210199 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:34.210706 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:34.210712 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:34.210997 (+ 285us) client_negotiation.cc:769] Sending connection context
0925 09:26:34.211270 (+ 273us) client_negotiation.cc:240] Negotiation successful
0925 09:26:34.211708 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":512,"threads_started":1}
I20260925 09:26:34.214696 11595 heartbeater.cc:347] Connected to a master server at 127.10.114.126:39305
I20260925 09:26:34.214988 11595 heartbeater.cc:464] Registering TS with master...
I20260925 09:26:34.215812 11595 heartbeater.cc:511] Master 127.10.114.126:39305 requested a full tablet report, sending...
I20260925 09:26:34.217242 11272 ts_manager.cc:194] Registered new tserver with Master: 0207ee4d84514b2e843a30e65f15ff9f (127.10.114.66:45487)
I20260925 09:26:34.218043 11272 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:34875
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
I20260925 09:26:34.232697 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39305
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:39655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:34.542225 11600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:34.542516 11600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:34.542712 11600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:34.549520 11600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:34.549669 11600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:34.549854 11600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:26:34.558983 11600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:39655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39305
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:34.560165 11600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:34.561513 11600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:34.573781 11606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:34.573829 11605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:34.573850 11608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:34.574712 11600 server_base.cc:1034] running on GCE node
I20260925 09:26:34.575362 11600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:34.576380 11600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:34.577606 11600 hybrid_clock.cc:648] HybridClock initialized: now 1790328394577564 us; error 63 us; skew 500 ppm
I20260925 09:26:34.577908 11600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:34 dist-test-slave-9pj8 krb5kdc[11227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328394, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:34.580961 11600 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:26:34.582424 11600 webserver.cc:460] Webserver started at http://127.10.114.67:34187/ using document root <none> and password file <none>
I20260925 09:26:34.583069 11600 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:34.583211 11600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:34.583537 11600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:34.586299 11600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "fcddfc7f40bc49da9baf6075a93cd54b"
format_stamp: "Formatted at 2026-09-25 09:26:34 on dist-test-slave-9pj8"
server_key: "adbc9e747a056e89dce16dacff5d475a"
server_key_iv: "8fcf01af2aff24637d78c1d07ebeaa09"
server_key_version: "encryptionkey@0"
I20260925 09:26:34.586993 11600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "fcddfc7f40bc49da9baf6075a93cd54b"
format_stamp: "Formatted at 2026-09-25 09:26:34 on dist-test-slave-9pj8"
server_key: "adbc9e747a056e89dce16dacff5d475a"
server_key_iv: "8fcf01af2aff24637d78c1d07ebeaa09"
server_key_version: "encryptionkey@0"
I20260925 09:26:34.591687 11600 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260925 09:26:34.595090 11615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:34.596517 11600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260925 09:26:34.596705 11600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "fcddfc7f40bc49da9baf6075a93cd54b"
format_stamp: "Formatted at 2026-09-25 09:26:34 on dist-test-slave-9pj8"
server_key: "adbc9e747a056e89dce16dacff5d475a"
server_key_iv: "8fcf01af2aff24637d78c1d07ebeaa09"
server_key_version: "encryptionkey@0"
I20260925 09:26:34.596935 11600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:34.611532 11600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:34.615269 11600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:34.615554 11600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:34.616618 11600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:26:34.617970 11600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:26:34.618077 11600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:34.618222 11600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:26:34.618302 11600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:34.653517 11600 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:41061
I20260925 09:26:34.653546 11727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:41061 every 8 connection(s)
I20260925 09:26:34.654750 11600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:26:34.656284 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 11600
I20260925 09:26:34.656461 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:26:34.656898 10697 external_mini_cluster.cc:1428] Setting key 8796b45e502f44a3f6cb4786d5776d70
Sep 25 09:26:34 dist-test-slave-9pj8 krb5kdc[11227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328394, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:34.674050 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:34.657282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:33111 (local address 127.10.114.126:39305)
0925 09:26:34.657474 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:34.657480 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:34.658816 (+ 1336us) server_negotiation.cc:407] Connection header received
0925 09:26:34.660683 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:34.660688 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:34.660773 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:34.660874 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:34.662742 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.663435 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.664924 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.665316 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.665441 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:34.669012 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:34.669023 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:34.669026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:34.669055 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:34.671677 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:34.672340 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:34.672344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:34.672346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:34.672453 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:34.672813 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:34.672816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:34.672818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:34.673047 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:34.673161 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:34.673704 (+ 543us) server_negotiation.cc:299] Negotiation successful
0925 09:26:34.673847 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 09:26:34.674944 11730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:34.657829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39305 (local address 127.10.114.67:33111)
0925 09:26:34.658680 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:34.658725 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:34.660426 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:34.661125 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:34.661136 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:34.661870 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:34.662594 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:34.662611 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.663580 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.663585 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:34.664788 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:34.664799 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.665538 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.665547 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:34.665801 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:34.666409 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:34.666429 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:34.668795 (+ 2366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:34.671851 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:34.671859 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:34.671878 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:34.672233 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:34.672547 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:34.672550 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:34.672552 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:34.672715 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:34.673157 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:34.673162 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:34.673425 (+ 263us) client_negotiation.cc:769] Sending connection context
0925 09:26:34.673620 (+ 195us) client_negotiation.cc:240] Negotiation successful
0925 09:26:34.673971 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":335,"threads_started":1}
I20260925 09:26:34.676246 11728 heartbeater.cc:347] Connected to a master server at 127.10.114.126:39305
I20260925 09:26:34.676486 11728 heartbeater.cc:464] Registering TS with master...
I20260925 09:26:34.677062 11728 heartbeater.cc:511] Master 127.10.114.126:39305 requested a full tablet report, sending...
I20260925 09:26:34.678301 11272 ts_manager.cc:194] Registered new tserver with Master: fcddfc7f40bc49da9baf6075a93cd54b (127.10.114.67:41061)
I20260925 09:26:34.679134 11272 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:33111
I20260925 09:26:34.683768 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:26:34.701671 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:34.689689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53544 (local address 127.10.114.126:39305)
0925 09:26:34.689830 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:34.689835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:34.690404 (+ 569us) server_negotiation.cc:407] Connection header received
0925 09:26:34.690517 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:34.690520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:34.690583 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:34.690672 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:34.692132 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.692829 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.694395 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.694835 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.694941 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:34.697069 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:34.697074 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:34.697077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:34.697105 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:34.699377 (+ 2272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:34.700002 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:34.700006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:34.700008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:34.700119 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:34.700440 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:34.700443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:34.700445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:34.700742 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:34.700911 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:34.701299 (+ 388us) server_negotiation.cc:299] Negotiation successful
0925 09:26:34.701458 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 09:26:34.707393 11272 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:26:34.713748 11272 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:26:34.748329 11461 heartbeater.cc:503] Master 127.10.114.126:39305 was elected leader, sending a full tablet report...
I20260925 09:26:34.753250 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:34.725779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42979 (local address 127.0.0.1:59276)
0925 09:26:34.735468 (+ 9689us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:34.735506 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:34.735690 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:34.742333 (+ 6643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:34.742343 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:34.742394 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:34.742867 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:34.742881 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.745575 (+ 2694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.745582 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:34.749419 (+ 3837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:34.749450 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.752127 (+ 2677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.752136 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:34.752390 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:34.752459 (+ 69us) client_negotiation.cc:769] Sending connection context
0925 09:26:34.752584 (+ 125us) client_negotiation.cc:240] Negotiation successful
0925 09:26:34.752815 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9426,"thread_start_us":334,"threads_started":1}
I20260925 09:26:34.754009 11741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:34.726113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59276 (local address 127.10.114.65:42979)
0925 09:26:34.741414 (+ 15301us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:34.741426 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:34.741459 (+ 33us) server_negotiation.cc:407] Connection header received
0925 09:26:34.741568 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:34.741578 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:34.741859 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:34.743114 (+ 1255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:34.743449 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.745380 (+ 1931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.749688 (+ 4308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.751643 (+ 1955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.753247 (+ 1604us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:34.753494 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:34.753562 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 09:26:34.753720 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15097,"thread_start_us":15905,"threads_started":1}
I20260925 09:26:34.757797 11396 tablet_service.cc:1467] Processing CreateTablet for tablet 94449e003a8a4f7fbcec96e99a81c054 (DEFAULT_TABLE table=test-table [id=7264ec32ce74472ca1e0dc959dac77a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:34.759119 11396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94449e003a8a4f7fbcec96e99a81c054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:34.761200 11742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:34.726425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:41061 (local address 127.0.0.1:48730)
0925 09:26:34.752746 (+ 26321us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:34.752763 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:34.752860 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:34.754615 (+ 1755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:34.754621 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:34.754649 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:34.755054 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:34.755065 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.756860 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.756865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:34.758204 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:34.758214 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.760326 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.760332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:34.760489 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:34.760519 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 09:26:34.760628 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 09:26:34.760817 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26218,"mutex_wait_us":3,"thread_start_us":26110,"threads_started":1}
I20260925 09:26:34.761518 11745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:34.727228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48446 (local address 127.10.114.66:45487)
0925 09:26:34.751086 (+ 23858us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:34.751096 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:34.753729 (+ 2633us) server_negotiation.cc:407] Connection header received
0925 09:26:34.753858 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:34.753865 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:34.754075 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:34.754356 (+ 281us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:34.754881 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.756495 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.759079 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.760427 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.760607 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:34.760815 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:34.761015 (+ 200us) server_negotiation.cc:299] Negotiation successful
0925 09:26:34.761162 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23631,"thread_start_us":23858,"threads_started":1}
I20260925 09:26:34.761801 11743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:34.726665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48730 (local address 127.10.114.67:41061)
0925 09:26:34.753833 (+ 27168us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:34.753840 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:34.753875 (+ 35us) server_negotiation.cc:407] Connection header received
0925 09:26:34.753943 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:34.753947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:34.754182 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:34.754349 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:34.755243 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.756714 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.758346 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.760132 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.760257 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:34.760450 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:34.761310 (+ 860us) server_negotiation.cc:299] Negotiation successful
0925 09:26:34.761482 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25585,"thread_start_us":26540,"threads_started":1}
I20260925 09:26:34.761843 11744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:34.727013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:45487 (local address 127.0.0.1:48446)
0925 09:26:34.753615 (+ 26602us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:34.753632 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:34.753738 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:34.754338 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:34.754343 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:34.754367 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:34.754688 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:34.754704 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.756624 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.756628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:34.758542 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:34.758555 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:34.760660 (+ 2105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:34.760666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:34.760851 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:34.760882 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 09:26:34.760970 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 09:26:34.761091 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26467,"thread_start_us":26016,"threads_started":1}
I20260925 09:26:34.764700 11530 tablet_service.cc:1467] Processing CreateTablet for tablet 94449e003a8a4f7fbcec96e99a81c054 (DEFAULT_TABLE table=test-table [id=7264ec32ce74472ca1e0dc959dac77a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:34.765090 11663 tablet_service.cc:1467] Processing CreateTablet for tablet 94449e003a8a4f7fbcec96e99a81c054 (DEFAULT_TABLE table=test-table [id=7264ec32ce74472ca1e0dc959dac77a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:34.765777 11530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94449e003a8a4f7fbcec96e99a81c054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:34.766290 11663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94449e003a8a4f7fbcec96e99a81c054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:34.773109 11746 tablet_bootstrap.cc:492] T 94449e003a8a4f7fbcec96e99a81c054 P 8a32dfaf4cfb44c684bb7d082e45f4f3: Bootstrap starting.
I20260925 09:26:34.776822 11747 tablet_bootstrap.cc:492] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f: Bootstrap starting.
I20260925 09:26:34.777838 11746 tablet_bootstrap.cc:654] T 94449e003a8a4f7fbcec96e99a81c054 P 8a32dfaf4cfb44c684bb7d082e45f4f3: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:34.778731 11746 log.cc:824] T 94449e003a8a4f7fbcec96e99a81c054 P 8a32dfaf4cfb44c684bb7d082e45f4f3: Log is configured to *not* fsync() on all Append() calls
I20260925 09:26:34.780130 11747 tablet_bootstrap.cc:654] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:34.780730 11749 tablet_bootstrap.cc:492] T 94449e003a8a4f7fbcec96e99a81c054 P fcddfc7f40bc49da9baf6075a93cd54b: Bootstrap starting.
I20260925 09:26:34.781265 11747 log.cc:824] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f: Log is configured to *not* fsync() on all Append() calls
I20260925 09:26:34.781468 11746 tablet_bootstrap.cc:492] T 94449e003a8a4f7fbcec96e99a81c054 P 8a32dfaf4cfb44c684bb7d082e45f4f3: No bootstrap required, opened a new log
I20260925 09:26:34.781720 11746 ts_tablet_manager.cc:1397] T 94449e003a8a4f7fbcec96e99a81c054 P 8a32dfaf4cfb44c684bb7d082e45f4f3: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260925 09:26:34.784298 11749 tablet_bootstrap.cc:654] T 94449e003a8a4f7fbcec96e99a81c054 P fcddfc7f40bc49da9baf6075a93cd54b: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:34.784658 11747 tablet_bootstrap.cc:492] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f: No bootstrap required, opened a new log
I20260925 09:26:34.784915 11747 ts_tablet_manager.cc:1397] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260925 09:26:34.785462 11749 log.cc:824] T 94449e003a8a4f7fbcec96e99a81c054 P fcddfc7f40bc49da9baf6075a93cd54b: Log is configured to *not* fsync() on all Append() calls
I20260925 09:26:34.788863 11749 tablet_bootstrap.cc:492] T 94449e003a8a4f7fbcec96e99a81c054 P fcddfc7f40bc49da9baf6075a93cd54b: No bootstrap required, opened a new log
I20260925 09:26:34.789096 11749 ts_tablet_manager.cc:1397] T 94449e003a8a4f7fbcec96e99a81c054 P fcddfc7f40bc49da9baf6075a93cd54b: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260925 09:26:34.790315 11746 raft_consensus.cc:348] T 94449e003a8a4f7fbcec96e99a81c054 P 8a32dfaf4cfb44c684bb7d082e45f4f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a32dfaf4cfb44c684bb7d082e45f4f3" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42979 } } peers { permanent_uuid: "0207ee4d84514b2e843a30e65f15ff9f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 45487 } } peers { permanent_uuid: "fcddfc7f40bc49da9baf6075a93cd54b" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41061 } }
I20260925 09:26:34.790638 11746 raft_consensus.cc:374] T 94449e003a8a4f7fbcec96e99a81c054 P 8a32dfaf4cfb44c684bb7d082e45f4f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:34.790740 11746 raft_consensus.cc:729] T 94449e003a8a4f7fbcec96e99a81c054 P 8a32dfaf4cfb44c684bb7d082e45f4f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a32dfaf4cfb44c684bb7d082e45f4f3, State: Initialized, Role: FOLLOWER
I20260925 09:26:34.791282 11746 consensus_queue.cc:260] T 94449e003a8a4f7fbcec96e99a81c054 P 8a32dfaf4cfb44c684bb7d082e45f4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a32dfaf4cfb44c684bb7d082e45f4f3" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42979 } } peers { permanent_uuid: "0207ee4d84514b2e843a30e65f15ff9f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 45487 } } peers { permanent_uuid: "fcddfc7f40bc49da9baf6075a93cd54b" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41061 } }
I20260925 09:26:34.792729 11746 ts_tablet_manager.cc:1428] T 94449e003a8a4f7fbcec96e99a81c054 P 8a32dfaf4cfb44c684bb7d082e45f4f3: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260925 09:26:34.794790 11747 raft_consensus.cc:348] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a32dfaf4cfb44c684bb7d082e45f4f3" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42979 } } peers { permanent_uuid: "0207ee4d84514b2e843a30e65f15ff9f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 45487 } } peers { permanent_uuid: "fcddfc7f40bc49da9baf6075a93cd54b" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41061 } }
I20260925 09:26:34.795140 11747 raft_consensus.cc:374] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:34.795249 11747 raft_consensus.cc:729] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0207ee4d84514b2e843a30e65f15ff9f, State: Initialized, Role: FOLLOWER
I20260925 09:26:34.795840 11747 consensus_queue.cc:260] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a32dfaf4cfb44c684bb7d082e45f4f3" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42979 } } peers { permanent_uuid: "0207ee4d84514b2e843a30e65f15ff9f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 45487 } } peers { permanent_uuid: "fcddfc7f40bc49da9baf6075a93cd54b" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41061 } }
I20260925 09:26:34.796844 11595 heartbeater.cc:503] Master 127.10.114.126:39305 was elected leader, sending a full tablet report...
I20260925 09:26:34.797873 11747 ts_tablet_manager.cc:1428] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260925 09:26:34.799317 11749 raft_consensus.cc:348] T 94449e003a8a4f7fbcec96e99a81c054 P fcddfc7f40bc49da9baf6075a93cd54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a32dfaf4cfb44c684bb7d082e45f4f3" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42979 } } peers { permanent_uuid: "0207ee4d84514b2e843a30e65f15ff9f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 45487 } } peers { permanent_uuid: "fcddfc7f40bc49da9baf6075a93cd54b" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41061 } }
I20260925 09:26:34.799619 11749 raft_consensus.cc:374] T 94449e003a8a4f7fbcec96e99a81c054 P fcddfc7f40bc49da9baf6075a93cd54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:34.799775 11749 raft_consensus.cc:729] T 94449e003a8a4f7fbcec96e99a81c054 P fcddfc7f40bc49da9baf6075a93cd54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcddfc7f40bc49da9baf6075a93cd54b, State: Initialized, Role: FOLLOWER
I20260925 09:26:34.800349 11749 consensus_queue.cc:260] T 94449e003a8a4f7fbcec96e99a81c054 P fcddfc7f40bc49da9baf6075a93cd54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a32dfaf4cfb44c684bb7d082e45f4f3" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42979 } } peers { permanent_uuid: "0207ee4d84514b2e843a30e65f15ff9f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 45487 } } peers { permanent_uuid: "fcddfc7f40bc49da9baf6075a93cd54b" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41061 } }
I20260925 09:26:34.801271 11728 heartbeater.cc:503] Master 127.10.114.126:39305 was elected leader, sending a full tablet report...
I20260925 09:26:34.801940 11749 ts_tablet_manager.cc:1428] T 94449e003a8a4f7fbcec96e99a81c054 P fcddfc7f40bc49da9baf6075a93cd54b: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
W20260925 09:26:34.906340 11729 tablet.cc:2369] T 94449e003a8a4f7fbcec96e99a81c054 P fcddfc7f40bc49da9baf6075a93cd54b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:26:34.942224 11596 tablet.cc:2369] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:26:34.971407 11462 tablet.cc:2369] T 94449e003a8a4f7fbcec96e99a81c054 P 8a32dfaf4cfb44c684bb7d082e45f4f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:26:35.053140 11753 raft_consensus.cc:482] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:26:35.053404 11753 raft_consensus.cc:504] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a32dfaf4cfb44c684bb7d082e45f4f3" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42979 } } peers { permanent_uuid: "0207ee4d84514b2e843a30e65f15ff9f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 45487 } } peers { permanent_uuid: "fcddfc7f40bc49da9baf6075a93cd54b" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41061 } }
I20260925 09:26:35.055256 11753 leader_election.cc:288] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a32dfaf4cfb44c684bb7d082e45f4f3 (127.10.114.65:42979), fcddfc7f40bc49da9baf6075a93cd54b (127.10.114.67:41061)
I20260925 09:26:35.062206 11741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:35.055546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:40195 (local address 127.10.114.65:42979)
0925 09:26:35.055815 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:35.055822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:35.056557 (+ 735us) server_negotiation.cc:407] Connection header received
0925 09:26:35.056736 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:35.056739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:35.056792 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:35.056895 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:35.057488 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:35.058175 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:35.060509 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:35.061253 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:35.061409 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:35.061637 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:35.061808 (+ 171us) server_negotiation.cc:299] Negotiation successful
0925 09:26:35.061941 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 09:26:35.062274 11743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:35.055759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:40551 (local address 127.10.114.67:41061)
0925 09:26:35.055975 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:35.055982 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:35.056430 (+ 448us) server_negotiation.cc:407] Connection header received
0925 09:26:35.056688 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:35.056693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:35.056773 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:35.056838 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:35.057450 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:35.058183 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:35.060554 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:35.061270 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:35.061448 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:35.061593 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:35.061909 (+ 316us) server_negotiation.cc:299] Negotiation successful
0925 09:26:35.062043 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 09:26:35.062317 11756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:35.055579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42979 (local address 127.10.114.66:40195)
0925 09:26:35.056460 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:35.056474 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:35.056598 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:35.056936 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:35.056941 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:35.056970 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:35.057322 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:35.057330 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:35.058313 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:35.058320 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:35.060367 (+ 2047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:35.060378 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:35.061476 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:35.061483 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:35.061665 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:35.061688 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 09:26:35.061762 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 09:26:35.061888 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"mutex_wait_us":22,"thread_start_us":613,"threads_started":1}
I20260925 09:26:35.062362 11755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:35.055546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:41061 (local address 127.10.114.66:40551)
0925 09:26:35.056325 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:35.056340 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:35.056501 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:35.056883 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:35.056888 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:35.056909 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:35.057317 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:35.057325 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:35.058331 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:35.058334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:35.060379 (+ 2045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:35.060390 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:35.061516 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:35.061523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:35.061718 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:35.061762 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 09:26:35.061855 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 09:26:35.061965 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"mutex_wait_us":17,"thread_start_us":451,"threads_started":1}
I20260925 09:26:35.063254 11416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94449e003a8a4f7fbcec96e99a81c054" candidate_uuid: "0207ee4d84514b2e843a30e65f15ff9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a32dfaf4cfb44c684bb7d082e45f4f3" is_pre_election: true
I20260925 09:26:35.063253 11683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94449e003a8a4f7fbcec96e99a81c054" candidate_uuid: "0207ee4d84514b2e843a30e65f15ff9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcddfc7f40bc49da9baf6075a93cd54b" is_pre_election: true
I20260925 09:26:35.063716 11683 raft_consensus.cc:2454] T 94449e003a8a4f7fbcec96e99a81c054 P fcddfc7f40bc49da9baf6075a93cd54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0207ee4d84514b2e843a30e65f15ff9f in term 0.
I20260925 09:26:35.063733 11416 raft_consensus.cc:2454] T 94449e003a8a4f7fbcec96e99a81c054 P 8a32dfaf4cfb44c684bb7d082e45f4f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0207ee4d84514b2e843a30e65f15ff9f in term 0.
I20260925 09:26:35.064416 11486 leader_election.cc:302] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0207ee4d84514b2e843a30e65f15ff9f, fcddfc7f40bc49da9baf6075a93cd54b; no voters:
I20260925 09:26:35.064793 11753 raft_consensus.cc:2781] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:26:35.064931 11753 raft_consensus.cc:482] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:26:35.065037 11753 raft_consensus.cc:3037] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:35.067096 11753 raft_consensus.cc:504] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a32dfaf4cfb44c684bb7d082e45f4f3" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42979 } } peers { permanent_uuid: "0207ee4d84514b2e843a30e65f15ff9f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 45487 } } peers { permanent_uuid: "fcddfc7f40bc49da9baf6075a93cd54b" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41061 } }
I20260925 09:26:35.067651 11753 leader_election.cc:288] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f [CANDIDATE]: Term 1 election: Requested vote from peers 8a32dfaf4cfb44c684bb7d082e45f4f3 (127.10.114.65:42979), fcddfc7f40bc49da9baf6075a93cd54b (127.10.114.67:41061)
I20260925 09:26:35.068075 11683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94449e003a8a4f7fbcec96e99a81c054" candidate_uuid: "0207ee4d84514b2e843a30e65f15ff9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcddfc7f40bc49da9baf6075a93cd54b"
I20260925 09:26:35.068126 11416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94449e003a8a4f7fbcec96e99a81c054" candidate_uuid: "0207ee4d84514b2e843a30e65f15ff9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a32dfaf4cfb44c684bb7d082e45f4f3"
I20260925 09:26:35.068279 11683 raft_consensus.cc:3037] T 94449e003a8a4f7fbcec96e99a81c054 P fcddfc7f40bc49da9baf6075a93cd54b [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:35.068346 11416 raft_consensus.cc:3037] T 94449e003a8a4f7fbcec96e99a81c054 P 8a32dfaf4cfb44c684bb7d082e45f4f3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:35.071398 11416 raft_consensus.cc:2454] T 94449e003a8a4f7fbcec96e99a81c054 P 8a32dfaf4cfb44c684bb7d082e45f4f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0207ee4d84514b2e843a30e65f15ff9f in term 1.
I20260925 09:26:35.071465 11683 raft_consensus.cc:2454] T 94449e003a8a4f7fbcec96e99a81c054 P fcddfc7f40bc49da9baf6075a93cd54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0207ee4d84514b2e843a30e65f15ff9f in term 1.
I20260925 09:26:35.071818 11485 leader_election.cc:302] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0207ee4d84514b2e843a30e65f15ff9f, 8a32dfaf4cfb44c684bb7d082e45f4f3; no voters:
I20260925 09:26:35.072101 11753 raft_consensus.cc:2781] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:26:35.072508 11753 raft_consensus.cc:686] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f [term 1 LEADER]: Becoming Leader. State: Replica: 0207ee4d84514b2e843a30e65f15ff9f, State: Running, Role: LEADER
I20260925 09:26:35.073033 11753 consensus_queue.cc:237] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a32dfaf4cfb44c684bb7d082e45f4f3" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42979 } } peers { permanent_uuid: "0207ee4d84514b2e843a30e65f15ff9f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 45487 } } peers { permanent_uuid: "fcddfc7f40bc49da9baf6075a93cd54b" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41061 } }
I20260925 09:26:35.077821 11272 catalog_manager.cc:5495] T 94449e003a8a4f7fbcec96e99a81c054 P 0207ee4d84514b2e843a30e65f15ff9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0207ee4d84514b2e843a30e65f15ff9f (127.10.114.66). New cstate: current_term: 1 leader_uuid: "0207ee4d84514b2e843a30e65f15ff9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a32dfaf4cfb44c684bb7d082e45f4f3" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0207ee4d84514b2e843a30e65f15ff9f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 45487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcddfc7f40bc49da9baf6075a93cd54b" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
Sep 25 09:26:35 dist-test-slave-9pj8 krb5kdc[11227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:26:35.103478 11741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:35.089061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59288 (local address 127.10.114.65:42979)
0925 09:26:35.089230 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:35.089234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:35.089704 (+ 470us) server_negotiation.cc:407] Connection header received
0925 09:26:35.089895 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:35.089900 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:35.089957 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:35.090055 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:35.091410 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:35.092213 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:35.093765 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:35.094159 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:35.094281 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:35.097960 (+ 3679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:35.097972 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:35.097987 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:35.098019 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:35.101163 (+ 3144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:35.101769 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:35.101777 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:35.101779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:35.101925 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:35.102262 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:35.102267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:35.102269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:35.102637 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:35.102853 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:35.103059 (+ 206us) server_negotiation.cc:299] Negotiation successful
0925 09:26:35.103224 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
Sep 25 09:26:35 dist-test-slave-9pj8 krb5kdc[11227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328395, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 09:26:35 dist-test-slave-9pj8 krb5kdc[11227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328395, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:26:35.148484 11741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:35.132999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59292 (local address 127.10.114.65:42979)
0925 09:26:35.133254 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:35.133264 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:35.133579 (+ 315us) server_negotiation.cc:407] Connection header received
0925 09:26:35.133838 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:35.133843 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:35.133938 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:35.134042 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:35.135697 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:35.136876 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:35.138070 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:35.138756 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:35.138886 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:35.142880 (+ 3994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:35.142888 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:35.142893 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:35.142931 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:35.146058 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:35.146596 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:35.146603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:35.146607 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:35.146746 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:35.147091 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:35.147096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:35.147100 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:35.147419 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:35.147719 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:35.147884 (+ 165us) server_negotiation.cc:299] Negotiation successful
0925 09:26:35.148096 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
Sep 25 09:26:35 dist-test-slave-9pj8 krb5kdc[11227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 09:26:35 dist-test-slave-9pj8 krb5kdc[11227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:26:35.191558 11741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:35.177681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59302 (local address 127.10.114.65:42979)
0925 09:26:35.177874 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:35.177883 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:35.178196 (+ 313us) server_negotiation.cc:407] Connection header received
0925 09:26:35.178441 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:35.178446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:35.178506 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:35.178608 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:35.180323 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:35.180995 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:35.182284 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:35.182683 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:35.182814 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:35.186377 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:35.186382 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:35.186385 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:35.186419 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:35.188874 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:35.189557 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:35.189564 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:35.189568 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:35.189695 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:35.190056 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:35.190064 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:35.190067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:35.190371 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:35.190619 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:35.190949 (+ 330us) server_negotiation.cc:299] Negotiation successful
0925 09:26:35.191212 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260925 09:26:35.193349 11376 tablet_service.cc:2908] Rejecting scan request for tablet 94449e003a8a4f7fbcec96e99a81c054: Uninitialized: clean time has not yet been initialized
I20260925 09:26:35.196133 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 11333
I20260925 09:26:35.211854 11741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:35.197486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59312 (local address 127.10.114.65:42979)
0925 09:26:35.197633 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:35.197637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:35.200236 (+ 2599us) server_negotiation.cc:407] Connection header received
0925 09:26:35.200465 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:35.200469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:35.200537 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:35.200652 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:35.201941 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:35.202639 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:35.204360 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:35.204774 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:35.204932 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:35.207051 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:35.207056 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:35.207059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:35.207103 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:35.209392 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:35.210259 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:35.210267 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:35.210269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:35.210411 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:35.210769 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:35.210772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:35.210774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:35.211002 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:35.211219 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:35.211389 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 09:26:35.211524 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 09:26:35.212431 11456 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:26:35.298527 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 11333
I20260925 09:26:35.315853 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 11467
Sep 25 09:26:35 dist-test-slave-9pj8 krb5kdc[11227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.66@KRBTEST.COM
I20260925 09:26:35.330029 11771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:35.316673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48460 (local address 127.10.114.66:45487)
0925 09:26:35.317137 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:35.317141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:35.317157 (+ 16us) server_negotiation.cc:407] Connection header received
0925 09:26:35.317218 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:35.317221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:35.317273 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:35.317357 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:35.318549 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:35.319215 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:35.320678 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:35.321064 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:35.321164 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:35.324952 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:35.324964 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:35.324979 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:35.325024 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:35.327436 (+ 2412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:35.328163 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:35.328171 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:35.328174 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:35.328322 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:35.328677 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:35.328682 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:35.328684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:35.329060 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:35.329289 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:35.329469 (+ 180us) server_negotiation.cc:299] Negotiation successful
0925 09:26:35.329613 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":239,"threads_started":1}
I20260925 09:26:35.330607 11590 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:26:35.416437 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 11467
I20260925 09:26:35.432627 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 11600
Sep 25 09:26:35 dist-test-slave-9pj8 krb5kdc[11227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.67@KRBTEST.COM
I20260925 09:26:35.446336 11743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:35.433358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48744 (local address 127.10.114.67:41061)
0925 09:26:35.433572 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:35.433576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:35.433594 (+ 18us) server_negotiation.cc:407] Connection header received
0925 09:26:35.433737 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:35.433740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:35.433799 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:35.433879 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:35.435051 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:35.435735 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:35.436921 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:35.437274 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:35.437361 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:35.441233 (+ 3872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:35.441247 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:35.441262 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:35.441298 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:35.443982 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:35.444516 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:35.444523 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:35.444525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:35.444665 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:35.445040 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:35.445043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:35.445045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:35.445370 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:35.445589 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:35.445825 (+ 236us) server_negotiation.cc:299] Negotiation successful
0925 09:26:35.445968 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260925 09:26:35.446919 11723 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:26:35.530478 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 11600
I20260925 09:26:35.546332 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 11240
I20260925 09:26:35.547528 11302 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:26:35.652372 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 11240
2026-09-25T09:26:35Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4153 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:26:35 dist-test-slave-9pj8 krb5kdc[11777](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:26:35 dist-test-slave-9pj8 krb5kdc[11777](info): set up 2 sockets
Sep 25 09:26:35 dist-test-slave-9pj8 krb5kdc[11777](info): commencing operation
krb5kdc: starting...
W20260925 09:26:36.824290 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:26:36 dist-test-slave-9pj8 krb5kdc[11777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:26:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:26:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:26:36.996047 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:33243
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:41611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:33243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:37.297962 11790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:37.298247 11790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:37.298334 11790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:37.305119 11790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:26:37.305204 11790 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 09:26:37.305250 11790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:37.305310 11790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:37.305358 11790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:26:37.305416 11790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:26:37.314499 11790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:41611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:33243
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:33243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:37.315578 11790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:37.316757 11790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:37.328204 11795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:37.328285 11798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:37.328305 11796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:37.328946 11790 server_base.cc:1034] running on GCE node
I20260925 09:26:37.329661 11790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:37.330933 11790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:37.332192 11790 hybrid_clock.cc:648] HybridClock initialized: now 1790328397332112 us; error 40 us; skew 500 ppm
I20260925 09:26:37.332518 11790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:37 dist-test-slave-9pj8 krb5kdc[11777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328397, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:37.340991 11790 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:26:37.342478 11790 webserver.cc:460] Webserver started at http://127.10.114.126:43133/ using document root <none> and password file <none>
I20260925 09:26:37.343156 11790 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:37.343290 11790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:37.343593 11790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:37.346406 11790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "993396ef0d824de1afaaa25ef205204a"
format_stamp: "Formatted at 2026-09-25 09:26:37 on dist-test-slave-9pj8"
server_key: "f9c779641d87b47b8c7b2d8cf5ddc7ba"
server_key_iv: "d34f58b889d58ed715a21ca35830ce23"
server_key_version: "encryptionkey@0"
I20260925 09:26:37.347128 11790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "993396ef0d824de1afaaa25ef205204a"
format_stamp: "Formatted at 2026-09-25 09:26:37 on dist-test-slave-9pj8"
server_key: "f9c779641d87b47b8c7b2d8cf5ddc7ba"
server_key_iv: "d34f58b889d58ed715a21ca35830ce23"
server_key_version: "encryptionkey@0"
I20260925 09:26:37.351981 11790 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260925 09:26:37.355161 11805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:37.356377 11790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:26:37.356537 11790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "993396ef0d824de1afaaa25ef205204a"
format_stamp: "Formatted at 2026-09-25 09:26:37 on dist-test-slave-9pj8"
server_key: "f9c779641d87b47b8c7b2d8cf5ddc7ba"
server_key_iv: "d34f58b889d58ed715a21ca35830ce23"
server_key_version: "encryptionkey@0"
I20260925 09:26:37.356736 11790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:37.383315 11790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:37.386862 11790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:37.387104 11790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:37.407027 11790 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:33243
I20260925 09:26:37.407047 11866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:33243 every 8 connection(s)
I20260925 09:26:37.408283 11790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:26:37.410429 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 11790
I20260925 09:26:37.410648 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:26:37.411264 10697 external_mini_cluster.cc:1428] Setting key d3ed534e37ad9e51a65107a6dff7ed90
I20260925 09:26:37.411984 11867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:37.423205 11867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a: Bootstrap starting.
Sep 25 09:26:37 dist-test-slave-9pj8 krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:37.426818 11867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:37.427896 11867 log.cc:824] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a: Log is configured to *not* fsync() on all Append() calls
I20260925 09:26:37.430876 11867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a: No bootstrap required, opened a new log
I20260925 09:26:37.431706 11870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:37.413082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44894 (local address 127.10.114.126:33243)
0925 09:26:37.413796 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:37.413810 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:37.413845 (+ 35us) server_negotiation.cc:407] Connection header received
0925 09:26:37.415304 (+ 1459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:37.415323 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:37.415646 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:37.416055 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:37.417328 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:37.418582 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:37.419732 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:37.420383 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:37.420605 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:37.424684 (+ 4079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:37.424690 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:37.424702 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:37.424743 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:37.427533 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:37.428276 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:37.428281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:37.428286 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:37.428535 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:37.428894 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:37.428901 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:37.428908 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:37.429412 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:37.429626 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:37.430153 (+ 527us) server_negotiation.cc:299] Negotiation successful
0925 09:26:37.430605 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":338,"threads_started":1}
I20260925 09:26:37.438412 11867 raft_consensus.cc:348] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993396ef0d824de1afaaa25ef205204a" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33243 } }
I20260925 09:26:37.438737 11867 raft_consensus.cc:374] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:37.438843 11867 raft_consensus.cc:729] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 993396ef0d824de1afaaa25ef205204a, State: Initialized, Role: FOLLOWER
I20260925 09:26:37.439357 11867 consensus_queue.cc:260] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993396ef0d824de1afaaa25ef205204a" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33243 } }
I20260925 09:26:37.439582 11867 raft_consensus.cc:388] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:26:37.439715 11867 raft_consensus.cc:482] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:26:37.439884 11867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:37.441767 11867 raft_consensus.cc:504] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993396ef0d824de1afaaa25ef205204a" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33243 } }
I20260925 09:26:37.442341 11867 leader_election.cc:302] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 993396ef0d824de1afaaa25ef205204a; no voters:
I20260925 09:26:37.442899 11867 leader_election.cc:288] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:26:37.442996 11872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:26:37.443662 11872 raft_consensus.cc:686] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a [term 1 LEADER]: Becoming Leader. State: Replica: 993396ef0d824de1afaaa25ef205204a, State: Running, Role: LEADER
I20260925 09:26:37.444161 11872 consensus_queue.cc:237] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993396ef0d824de1afaaa25ef205204a" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33243 } }
I20260925 09:26:37.444847 11867 sys_catalog.cc:564] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:26:37.446851 11874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 993396ef0d824de1afaaa25ef205204a. Latest consensus state: current_term: 1 leader_uuid: "993396ef0d824de1afaaa25ef205204a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993396ef0d824de1afaaa25ef205204a" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33243 } } }
I20260925 09:26:37.446817 11873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "993396ef0d824de1afaaa25ef205204a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993396ef0d824de1afaaa25ef205204a" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33243 } } }
I20260925 09:26:37.447211 11874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a [sys.catalog]: This master's current role is: LEADER
I20260925 09:26:37.447276 11873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a [sys.catalog]: This master's current role is: LEADER
I20260925 09:26:37.449554 11878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:26:37.454414 11878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:26:37.462641 11878 catalog_manager.cc:1347] Generated new cluster ID: 3db6c475c54148f299612f6c324371db
I20260925 09:26:37.462750 11878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:26:37.489391 11878 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:26:37.490705 11878 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:26:37.500073 11878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a: Generated new TSK 0
I20260925 09:26:37.500847 11878 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
I20260925 09:26:37.545480 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:33243
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:41611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:37.850706 11893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:37.850947 11893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:37.851037 11893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:37.857882 11893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:37.857993 11893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:37.858178 11893 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 09:26:37.858287 11893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:26:37.867648 11893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:41611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.114.126:33243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:37.868789 11893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:37.870069 11893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:37.882467 11899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:37.882550 11898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:37.882507 11901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:37.883229 11893 server_base.cc:1034] running on GCE node
I20260925 09:26:37.883822 11893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:37.884660 11893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:37.885890 11893 hybrid_clock.cc:648] HybridClock initialized: now 1790328397885827 us; error 41 us; skew 500 ppm
I20260925 09:26:37.886168 11893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:37 dist-test-slave-9pj8 krb5kdc[11777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328397, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:37.889070 11893 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:26:37.890394 11893 webserver.cc:460] Webserver started at http://127.10.114.65:35293/ using document root <none> and password file <none>
I20260925 09:26:37.891045 11893 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:37.891153 11893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:37.891474 11893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:37.894259 11893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "28001e65b61246d0b966f30e03845a70"
format_stamp: "Formatted at 2026-09-25 09:26:37 on dist-test-slave-9pj8"
server_key: "66abba167afb7c5d9742f6658907dfc3"
server_key_iv: "5cd231e0c60468abe8fa8f5927dcaf83"
server_key_version: "encryptionkey@0"
I20260925 09:26:37.895051 11893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "28001e65b61246d0b966f30e03845a70"
format_stamp: "Formatted at 2026-09-25 09:26:37 on dist-test-slave-9pj8"
server_key: "66abba167afb7c5d9742f6658907dfc3"
server_key_iv: "5cd231e0c60468abe8fa8f5927dcaf83"
server_key_version: "encryptionkey@0"
I20260925 09:26:37.899757 11893 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260925 09:26:37.902912 11908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:37.904098 11893 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.003s
I20260925 09:26:37.904239 11893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "28001e65b61246d0b966f30e03845a70"
format_stamp: "Formatted at 2026-09-25 09:26:37 on dist-test-slave-9pj8"
server_key: "66abba167afb7c5d9742f6658907dfc3"
server_key_iv: "5cd231e0c60468abe8fa8f5927dcaf83"
server_key_version: "encryptionkey@0"
I20260925 09:26:37.904433 11893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:37.929381 11893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:37.933660 11893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:37.934054 11893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:37.936718 11893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:26:37.936831 11893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:37.936975 11893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:26:37.937070 11893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 09:26:37 dist-test-slave-9pj8 krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328397, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:37.962314 11917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:37.937217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:43351 (local address 127.10.114.126:33243)
0925 09:26:37.939917 (+ 2700us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:37.939924 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:37.939954 (+ 30us) server_negotiation.cc:407] Connection header received
0925 09:26:37.941424 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:37.941435 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:37.941549 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:37.941698 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:37.943601 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:37.944618 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:37.946727 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:37.947396 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:37.947539 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:37.953661 (+ 6122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:37.953668 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:37.953674 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:37.953716 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:37.957459 (+ 3743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:37.958751 (+ 1292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:37.958758 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:37.958764 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:37.958933 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:37.959701 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:37.959708 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:37.959712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:37.960109 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:37.960271 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:37.961565 (+ 1294us) server_negotiation.cc:299] Negotiation successful
0925 09:26:37.961832 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2524,"thread_start_us":2398,"threads_started":1}
I20260925 09:26:37.964814 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:37.937963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:33243 (local address 127.10.114.65:43351)
0925 09:26:37.939020 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:37.939111 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:37.941120 (+ 2009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:37.941886 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:37.941893 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:37.942620 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:37.943386 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:37.943403 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:37.944756 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:37.944761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:37.946589 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:37.946600 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:37.947692 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:37.947705 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:37.948130 (+ 425us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:37.949004 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:37.949046 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:37.953388 (+ 4342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:37.957665 (+ 4277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:37.957680 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:37.957704 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:37.958592 (+ 888us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:37.959078 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:37.959084 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:37.959092 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:37.959457 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:37.960287 (+ 830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:37.960301 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:37.960903 (+ 602us) client_negotiation.cc:769] Sending connection context
0925 09:26:37.961475 (+ 572us) client_negotiation.cc:240] Negotiation successful
0925 09:26:37.962686 (+ 1211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":421,"threads_started":1}
I20260925 09:26:37.981698 11893 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:39319
I20260925 09:26:37.981735 12028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:39319 every 8 connection(s)
I20260925 09:26:37.983043 11893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:26:37.985630 11917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:37.970812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:37997 (local address 127.10.114.126:33243)
0925 09:26:37.971055 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:37.971062 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:37.972073 (+ 1011us) server_negotiation.cc:407] Connection header received
0925 09:26:37.972275 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:37.972280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:37.972348 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:37.972442 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:37.973334 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:37.974249 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:37.977189 (+ 2940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:37.977664 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:37.977802 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:37.980073 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:37.980078 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:37.980081 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:37.980106 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:37.982825 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:37.983548 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:37.983553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:37.983556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:37.983679 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:37.984137 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:37.984142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:37.984145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:37.984474 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:37.984613 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:37.985201 (+ 588us) server_negotiation.cc:299] Negotiation successful
0925 09:26:37.985338 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260925 09:26:37.985471 12029 heartbeater.cc:347] Connected to a master server at 127.10.114.126:33243
I20260925 09:26:37.985836 12003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:37.970990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:33243 (local address 127.10.114.65:37997)
0925 09:26:37.971969 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:37.971994 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:37.972140 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:37.972460 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:37.972465 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:37.972902 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:37.973205 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:37.973213 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:37.974431 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:37.974439 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:37.976972 (+ 2533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:37.977010 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:37.977838 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:37.977844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:37.978090 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:37.978830 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:37.978858 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:37.979894 (+ 1036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:37.982969 (+ 3075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:37.982974 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:37.982978 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:37.983425 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:37.983807 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:37.983811 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:37.983819 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:37.983977 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:37.984706 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:37.984711 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:37.984958 (+ 247us) client_negotiation.cc:769] Sending connection context
0925 09:26:37.985086 (+ 128us) client_negotiation.cc:240] Negotiation successful
0925 09:26:37.985441 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"thread_start_us":638,"threads_started":1}
I20260925 09:26:37.986325 12029 heartbeater.cc:464] Registering TS with master...
I20260925 09:26:37.986922 12029 heartbeater.cc:511] Master 127.10.114.126:33243 requested a full tablet report, sending...
I20260925 09:26:37.988921 11822 ts_manager.cc:194] Registered new tserver with Master: 28001e65b61246d0b966f30e03845a70 (127.10.114.65:39319)
I20260925 09:26:37.990260 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 11893
I20260925 09:26:37.990285 11822 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:43351
I20260925 09:26:37.990480 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:26:37.991103 10697 external_mini_cluster.cc:1428] Setting key 4c81903c50d15677bd68dc4fa32df5e9
WARNING: no policy specified for kudu/127.10.114.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
I20260925 09:26:38.030615 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:33243
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:41611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:38.338184 12033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:38.338449 12033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:38.338617 12033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:38.346117 12033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:38.346230 12033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:38.346379 12033 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 09:26:38.346505 12033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:26:38.355471 12033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:41611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.114.126:33243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:38.356642 12033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:38.357864 12033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:38.369915 12038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:38.369992 12039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:38.369961 12041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:38.370903 12033 server_base.cc:1034] running on GCE node
I20260925 09:26:38.371423 12033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:38.372291 12033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:38.373492 12033 hybrid_clock.cc:648] HybridClock initialized: now 1790328398373441 us; error 33 us; skew 500 ppm
I20260925 09:26:38.373766 12033 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:38 dist-test-slave-9pj8 krb5kdc[11777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328398, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:38.377070 12033 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:26:38.378455 12033 webserver.cc:460] Webserver started at http://127.10.114.66:39729/ using document root <none> and password file <none>
I20260925 09:26:38.379109 12033 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:38.379217 12033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:38.379498 12033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:38.382464 12033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "683276ca765b4e6aa49d33ffea085524"
format_stamp: "Formatted at 2026-09-25 09:26:38 on dist-test-slave-9pj8"
server_key: "af73c57354d59e8af82a1e2dc899a146"
server_key_iv: "d480d666a7e00e5f63a3d68a60cc224f"
server_key_version: "encryptionkey@0"
I20260925 09:26:38.383237 12033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "683276ca765b4e6aa49d33ffea085524"
format_stamp: "Formatted at 2026-09-25 09:26:38 on dist-test-slave-9pj8"
server_key: "af73c57354d59e8af82a1e2dc899a146"
server_key_iv: "d480d666a7e00e5f63a3d68a60cc224f"
server_key_version: "encryptionkey@0"
I20260925 09:26:38.388206 12033 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260925 09:26:38.391407 12048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:38.392772 12033 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:26:38.392906 12033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "683276ca765b4e6aa49d33ffea085524"
format_stamp: "Formatted at 2026-09-25 09:26:38 on dist-test-slave-9pj8"
server_key: "af73c57354d59e8af82a1e2dc899a146"
server_key_iv: "d480d666a7e00e5f63a3d68a60cc224f"
server_key_version: "encryptionkey@0"
I20260925 09:26:38.393095 12033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:38.406057 12033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:38.410526 12033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:38.410794 12033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:38.413434 12033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:26:38.413549 12033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:38.413697 12033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:26:38.413770 12033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 09:26:38 dist-test-slave-9pj8 krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328398, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:38.443249 11917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:38.416838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:45935 (local address 127.10.114.126:33243)
0925 09:26:38.417031 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:38.417040 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:38.418406 (+ 1366us) server_negotiation.cc:407] Connection header received
0925 09:26:38.422088 (+ 3682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:38.422098 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:38.422202 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:38.422335 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:38.424986 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:38.426516 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:38.429851 (+ 3335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:38.430374 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:38.430535 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:38.435635 (+ 5100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:38.435643 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:38.435650 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:38.435702 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:38.438992 (+ 3290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:38.440192 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:38.440201 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:38.440208 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:38.440395 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:38.440979 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:38.440985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:38.440990 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:38.441450 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:38.441633 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:38.442596 (+ 963us) server_negotiation.cc:299] Negotiation successful
0925 09:26:38.442824 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 09:26:38.444968 12059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:38.417063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:33243 (local address 127.10.114.66:45935)
0925 09:26:38.418185 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:38.418251 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:38.421590 (+ 3339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:38.422672 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:38.422688 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:38.423645 (+ 957us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:38.424714 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:38.424751 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:38.426768 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:38.426780 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:38.429648 (+ 2868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:38.429668 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:38.430663 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:38.430672 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:38.431037 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:38.431888 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:38.431936 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:38.435284 (+ 3348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:38.439164 (+ 3880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:38.439174 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:38.439196 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:38.439971 (+ 775us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:38.440557 (+ 586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:38.440578 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:38.440581 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:38.440836 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:38.441633 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:38.441642 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:38.442145 (+ 503us) client_negotiation.cc:769] Sending connection context
0925 09:26:38.442484 (+ 339us) client_negotiation.cc:240] Negotiation successful
0925 09:26:38.443049 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"thread_start_us":5307,"threads_started":1}
I20260925 09:26:38.462071 12033 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:40095
I20260925 09:26:38.462121 12167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:40095 every 8 connection(s)
I20260925 09:26:38.463593 12033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:26:38.465688 12168 heartbeater.cc:347] Connected to a master server at 127.10.114.126:33243
I20260925 09:26:38.465914 12168 heartbeater.cc:464] Registering TS with master...
I20260925 09:26:38.466372 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 12033
I20260925 09:26:38.466553 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:26:38.466679 12168 heartbeater.cc:511] Master 127.10.114.126:33243 requested a full tablet report, sending...
I20260925 09:26:38.467067 10697 external_mini_cluster.cc:1428] Setting key 8559ef597effb4a0d2003407e2b38b6c
I20260925 09:26:38.468375 11917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:38.451837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:50673 (local address 127.10.114.126:33243)
0925 09:26:38.452031 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:38.452039 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:38.453124 (+ 1085us) server_negotiation.cc:407] Connection header received
0925 09:26:38.453295 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:38.453300 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:38.453367 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:38.453466 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:38.454565 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:38.455749 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:38.458395 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:38.459142 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:38.459290 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:38.462037 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:38.462044 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:38.462051 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:38.462090 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:38.465381 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:38.466102 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:38.466109 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:38.466112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:38.466269 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:38.466764 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:38.466772 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:38.466776 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:38.467072 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:38.467266 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:38.467688 (+ 422us) server_negotiation.cc:299] Negotiation successful
0925 09:26:38.467993 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 09:26:38.468389 12143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:38.452219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:33243 (local address 127.10.114.66:50673)
0925 09:26:38.452993 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:38.453023 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:38.453165 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:38.453497 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:38.453504 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:38.453910 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:38.454394 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:38.454412 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:38.455952 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:38.455960 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:38.458202 (+ 2242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:38.458221 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:38.459368 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:38.459379 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:38.459596 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:38.460327 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:38.460343 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:38.461845 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:38.465501 (+ 3656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:38.465507 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:38.465511 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:38.466004 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:38.466411 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:38.466415 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:38.466419 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:38.466579 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:38.467239 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:38.467244 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:38.467436 (+ 192us) client_negotiation.cc:769] Sending connection context
0925 09:26:38.467596 (+ 160us) client_negotiation.cc:240] Negotiation successful
0925 09:26:38.467894 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":430,"threads_started":1}
I20260925 09:26:38.469496 11821 ts_manager.cc:194] Registered new tserver with Master: 683276ca765b4e6aa49d33ffea085524 (127.10.114.66:40095)
I20260925 09:26:38.470378 11821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:45935
WARNING: no policy specified for kudu/127.10.114.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
I20260925 09:26:38.505256 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:33243
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:41611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:38.804961 12172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:38.805218 12172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:38.805307 12172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:38.812237 12172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:38.812337 12172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:38.812415 12172 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 09:26:38.812491 12172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:26:38.821591 12172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:41611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.114.126:33243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:38.822659 12172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:38.823822 12172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:38.836160 12177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:38.836354 12178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:38.837044 12180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:38.837455 12172 server_base.cc:1034] running on GCE node
I20260925 09:26:38.838078 12172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:38.838845 12172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:38.840051 12172 hybrid_clock.cc:648] HybridClock initialized: now 1790328398840004 us; error 28 us; skew 500 ppm
I20260925 09:26:38.840341 12172 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:38 dist-test-slave-9pj8 krb5kdc[11777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328398, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:38.843338 12172 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:26:38.844732 12172 webserver.cc:460] Webserver started at http://127.10.114.67:32931/ using document root <none> and password file <none>
I20260925 09:26:38.845412 12172 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:38.845525 12172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:38.845826 12172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:38.848776 12172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "639a73312c9147aab1d84d88cd9d7293"
format_stamp: "Formatted at 2026-09-25 09:26:38 on dist-test-slave-9pj8"
server_key: "83ef2ebcb96a580ab9e8af7eca875cdd"
server_key_iv: "6cbac969420c40b4f8ac02de5ba16461"
server_key_version: "encryptionkey@0"
I20260925 09:26:38.849511 12172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "639a73312c9147aab1d84d88cd9d7293"
format_stamp: "Formatted at 2026-09-25 09:26:38 on dist-test-slave-9pj8"
server_key: "83ef2ebcb96a580ab9e8af7eca875cdd"
server_key_iv: "6cbac969420c40b4f8ac02de5ba16461"
server_key_version: "encryptionkey@0"
I20260925 09:26:38.854216 12172 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:26:38.857513 12187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:38.858907 12172 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 09:26:38.859083 12172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "639a73312c9147aab1d84d88cd9d7293"
format_stamp: "Formatted at 2026-09-25 09:26:38 on dist-test-slave-9pj8"
server_key: "83ef2ebcb96a580ab9e8af7eca875cdd"
server_key_iv: "6cbac969420c40b4f8ac02de5ba16461"
server_key_version: "encryptionkey@0"
I20260925 09:26:38.859328 12172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:38.872236 12172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:38.876262 12172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:38.876523 12172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:38.879033 12172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:26:38.879128 12172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:38.879247 12172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:26:38.879312 12172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 09:26:38 dist-test-slave-9pj8 krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328398, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:38.903510 11917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:38.879669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:41109 (local address 127.10.114.126:33243)
0925 09:26:38.879907 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:38.879916 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:38.881379 (+ 1463us) server_negotiation.cc:407] Connection header received
0925 09:26:38.883474 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:38.883482 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:38.883584 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:38.883689 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:38.886103 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:38.887223 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:38.889607 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:38.890121 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:38.890254 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:38.896038 (+ 5784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:38.896047 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:38.896053 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:38.896099 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:38.899913 (+ 3814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:38.900702 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:38.900710 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:38.900716 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:38.900906 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:38.901366 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:38.901374 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:38.901380 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:38.901817 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:38.902003 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:38.902707 (+ 704us) server_negotiation.cc:299] Negotiation successful
0925 09:26:38.903014 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260925 09:26:38.904040 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:38.880326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:33243 (local address 127.10.114.67:41109)
0925 09:26:38.881168 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:38.881209 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:38.883199 (+ 1990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:38.884264 (+ 1065us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:38.884272 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:38.884999 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:38.885899 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:38.885926 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:38.887369 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:38.887374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:38.889466 (+ 2092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:38.889477 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:38.890613 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:38.890622 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:38.890972 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:38.892030 (+ 1058us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:38.892059 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:38.895754 (+ 3695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:38.900078 (+ 4324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:38.900085 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:38.900095 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:38.900552 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:38.901020 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:38.901023 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:38.901025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:38.901220 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:38.901987 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:38.901993 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:38.902289 (+ 296us) client_negotiation.cc:769] Sending connection context
0925 09:26:38.902550 (+ 261us) client_negotiation.cc:240] Negotiation successful
0925 09:26:38.902885 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":385,"threads_started":1}
I20260925 09:26:38.925486 11917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:38.909821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:54063 (local address 127.10.114.126:33243)
0925 09:26:38.910051 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:38.910059 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:38.910622 (+ 563us) server_negotiation.cc:407] Connection header received
0925 09:26:38.910821 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:38.910827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:38.910904 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:38.911018 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:38.912536 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:38.913696 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:38.916104 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:38.916779 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:38.916902 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:38.918794 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:38.918802 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:38.918808 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:38.918848 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:38.922514 (+ 3666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:38.923340 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:38.923354 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:38.923360 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:38.923516 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:38.923942 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:38.923946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:38.923949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:38.924213 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:38.924385 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:38.924817 (+ 432us) server_negotiation.cc:299] Negotiation successful
0925 09:26:38.925075 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 09:26:38.925629 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:38.909792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:33243 (local address 127.10.114.67:54063)
0925 09:26:38.910511 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:38.910532 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:38.910670 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:38.911036 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:38.911051 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:38.911865 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:38.912346 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:38.912384 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:38.913901 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:38.913909 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:38.915917 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:38.915932 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:38.916956 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:38.916963 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:38.917161 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:38.917859 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:38.917873 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:38.918576 (+ 703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:38.922635 (+ 4059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:38.922642 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:38.922647 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:38.923207 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:38.923628 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:38.923632 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:38.923637 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:38.923817 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:38.924343 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:38.924346 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:38.924496 (+ 150us) client_negotiation.cc:769] Sending connection context
0925 09:26:38.924659 (+ 163us) client_negotiation.cc:240] Negotiation successful
0925 09:26:38.925019 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":349,"threads_started":1}
I20260925 09:26:38.926033 12172 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:46033
I20260925 09:26:38.926071 12306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:46033 every 8 connection(s)
I20260925 09:26:38.927491 12172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:26:38.928932 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 12172
I20260925 09:26:38.929118 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:26:38.929674 10697 external_mini_cluster.cc:1428] Setting key a9c504969340722093c28554e0ad76f7
I20260925 09:26:38.929536 12307 heartbeater.cc:347] Connected to a master server at 127.10.114.126:33243
I20260925 09:26:38.930617 12307 heartbeater.cc:464] Registering TS with master...
I20260925 09:26:38.931354 12307 heartbeater.cc:511] Master 127.10.114.126:33243 requested a full tablet report, sending...
I20260925 09:26:38.932772 11821 ts_manager.cc:194] Registered new tserver with Master: 639a73312c9147aab1d84d88cd9d7293 (127.10.114.67:46033)
I20260925 09:26:38.933574 11821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:41109
I20260925 09:26:38.944429 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 09:26:38 dist-test-slave-9pj8 krb5kdc[11777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328398, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 09:26:38 dist-test-slave-9pj8 krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328398, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:38.989693 11917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:38.975507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44904 (local address 127.10.114.126:33243)
0925 09:26:38.975745 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:38.975751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:38.975936 (+ 185us) server_negotiation.cc:407] Connection header received
0925 09:26:38.976146 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:38.976154 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:38.976218 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:38.976323 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:38.977623 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:38.978322 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:38.979648 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:38.980176 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:38.980315 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:38.983986 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:38.983991 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:38.983996 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:38.984033 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:38.987222 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:38.987892 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:38.987895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:38.987897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:38.988001 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:38.988322 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:38.988325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:38.988327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:38.988620 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:38.988791 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:38.989126 (+ 335us) server_negotiation.cc:299] Negotiation successful
0925 09:26:38.989397 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260925 09:26:38.993330 12029 heartbeater.cc:503] Master 127.10.114.126:33243 was elected leader, sending a full tablet report...
I20260925 09:26:38.993983 11821 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:26:39.000695 11821 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:26:39.038813 12322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:39.012508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:40095 (local address 127.0.0.1:45740)
0925 09:26:39.027838 (+ 15330us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:39.027912 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:39.028134 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:39.029423 (+ 1289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:39.029431 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:39.029460 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:39.029932 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:39.029942 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.032105 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.032113 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:39.034963 (+ 2850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:39.034981 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.037525 (+ 2544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.037534 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:39.037815 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:39.037880 (+ 65us) client_negotiation.cc:769] Sending connection context
0925 09:26:39.038026 (+ 146us) client_negotiation.cc:240] Negotiation successful
0925 09:26:39.038300 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14951,"thread_start_us":17442,"threads_started":1}
I20260925 09:26:39.039906 12321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:39.012621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45740 (local address 127.10.114.66:40095)
0925 09:26:39.028580 (+ 15959us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:39.028593 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:39.028633 (+ 40us) server_negotiation.cc:407] Connection header received
0925 09:26:39.028755 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:39.028766 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:39.029097 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:39.029333 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:39.030190 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.031899 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.035215 (+ 3316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.036912 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.037016 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:39.037257 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:39.039292 (+ 2035us) server_negotiation.cc:299] Negotiation successful
0925 09:26:39.039527 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15738,"thread_start_us":16924,"threads_started":1}
I20260925 09:26:39.041086 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:39.013242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:46033 (local address 127.0.0.1:32860)
0925 09:26:39.030282 (+ 17040us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:39.030301 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:39.030409 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:39.031455 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:39.031463 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:39.031496 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:39.032039 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:39.032054 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.034621 (+ 2567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.034629 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:39.037481 (+ 2852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:39.037503 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.039970 (+ 2467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.039979 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:39.040334 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:39.040383 (+ 49us) client_negotiation.cc:769] Sending connection context
0925 09:26:39.040460 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 09:26:39.040632 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16903,"thread_start_us":15107,"threads_started":1}
I20260925 09:26:39.041635 12323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:39.013637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32860 (local address 127.10.114.67:46033)
0925 09:26:39.029129 (+ 15492us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:39.029139 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:39.030543 (+ 1404us) server_negotiation.cc:407] Connection header received
0925 09:26:39.030639 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:39.030647 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:39.030902 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:39.031107 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:39.032294 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.034438 (+ 2144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.037737 (+ 3299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.039677 (+ 1940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.040838 (+ 1161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:39.041066 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:39.041145 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 09:26:39.041291 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15323,"thread_start_us":15493,"threads_started":1}
I20260925 09:26:39.045540 12098 tablet_service.cc:1467] Processing CreateTablet for tablet f249c6139d3d466283a6da8451ed0698 (DEFAULT_TABLE table=test-table [id=c2bea88744e54ae59e871f0d26ce2e18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:39.045555 12237 tablet_service.cc:1467] Processing CreateTablet for tablet f249c6139d3d466283a6da8451ed0698 (DEFAULT_TABLE table=test-table [id=c2bea88744e54ae59e871f0d26ce2e18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:39.046756 12237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f249c6139d3d466283a6da8451ed0698. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:39.046782 12098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f249c6139d3d466283a6da8451ed0698. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:39.049441 12322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:39.030889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:39319 (local address 127.0.0.1:60226)
0925 09:26:39.040509 (+ 9620us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:39.040530 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:39.040640 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:39.043371 (+ 2731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:39.043375 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:39.043398 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:39.043537 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:39.043546 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.045296 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.045300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:39.047021 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:39.047046 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.048778 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.048783 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:39.048954 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:39.048985 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 09:26:39.049074 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 09:26:39.049192 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9504,"thread_start_us":11439,"threads_started":1}
I20260925 09:26:39.049511 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:39.030661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60226 (local address 127.10.114.65:39319)
0925 09:26:39.042851 (+ 12190us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:39.042861 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:39.042888 (+ 27us) server_negotiation.cc:407] Connection header received
0925 09:26:39.042977 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:39.042984 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:39.043194 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:39.043371 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:39.043677 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.045173 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.047198 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.048599 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.048721 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:39.048899 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:39.049121 (+ 222us) server_negotiation.cc:299] Negotiation successful
0925 09:26:39.049254 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12051,"thread_start_us":11939,"threads_started":1}
I20260925 09:26:39.052824 11959 tablet_service.cc:1467] Processing CreateTablet for tablet f249c6139d3d466283a6da8451ed0698 (DEFAULT_TABLE table=test-table [id=c2bea88744e54ae59e871f0d26ce2e18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:39.053962 11959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f249c6139d3d466283a6da8451ed0698. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:39.057873 12326 tablet_bootstrap.cc:492] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293: Bootstrap starting.
I20260925 09:26:39.060144 12327 tablet_bootstrap.cc:492] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524: Bootstrap starting.
I20260925 09:26:39.061198 12326 tablet_bootstrap.cc:654] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:39.062600 12326 log.cc:824] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293: Log is configured to *not* fsync() on all Append() calls
I20260925 09:26:39.063795 12327 tablet_bootstrap.cc:654] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:39.065223 12327 log.cc:824] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524: Log is configured to *not* fsync() on all Append() calls
I20260925 09:26:39.065766 12329 tablet_bootstrap.cc:492] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70: Bootstrap starting.
I20260925 09:26:39.068423 12329 tablet_bootstrap.cc:654] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:39.069285 12329 log.cc:824] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70: Log is configured to *not* fsync() on all Append() calls
I20260925 09:26:39.071463 12326 tablet_bootstrap.cc:492] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293: No bootstrap required, opened a new log
I20260925 09:26:39.071630 12329 tablet_bootstrap.cc:492] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70: No bootstrap required, opened a new log
I20260925 09:26:39.071725 12326 ts_tablet_manager.cc:1397] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.006s
I20260925 09:26:39.071925 12329 ts_tablet_manager.cc:1397] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260925 09:26:39.073362 12327 tablet_bootstrap.cc:492] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524: No bootstrap required, opened a new log
I20260925 09:26:39.073653 12327 ts_tablet_manager.cc:1397] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260925 09:26:39.079010 12326 raft_consensus.cc:348] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.079331 12326 raft_consensus.cc:374] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:39.079456 12326 raft_consensus.cc:729] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639a73312c9147aab1d84d88cd9d7293, State: Initialized, Role: FOLLOWER
I20260925 09:26:39.080045 12326 consensus_queue.cc:260] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.081028 12307 heartbeater.cc:503] Master 127.10.114.126:33243 was elected leader, sending a full tablet report...
I20260925 09:26:39.081646 12326 ts_tablet_manager.cc:1428] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293: Time spent starting tablet: real 0.010s user 0.006s sys 0.003s
I20260925 09:26:39.082177 12329 raft_consensus.cc:348] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.082549 12329 raft_consensus.cc:374] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:39.082633 12329 raft_consensus.cc:729] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28001e65b61246d0b966f30e03845a70, State: Initialized, Role: FOLLOWER
I20260925 09:26:39.083142 12329 consensus_queue.cc:260] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.083700 12327 raft_consensus.cc:348] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.084082 12327 raft_consensus.cc:374] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:39.084185 12327 raft_consensus.cc:729] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 683276ca765b4e6aa49d33ffea085524, State: Initialized, Role: FOLLOWER
I20260925 09:26:39.084650 12329 ts_tablet_manager.cc:1428] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260925 09:26:39.084671 12327 consensus_queue.cc:260] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.086405 12168 heartbeater.cc:503] Master 127.10.114.126:33243 was elected leader, sending a full tablet report...
I20260925 09:26:39.086719 12327 ts_tablet_manager.cc:1428] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524: Time spent starting tablet: real 0.013s user 0.004s sys 0.008s
W20260925 09:26:39.180025 12308 tablet.cc:2369] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:26:39.215946 12169 tablet.cc:2369] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:26:39.235874 12030 tablet.cc:2369] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:26:39.274931 12333 raft_consensus.cc:482] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:26:39.275285 12333 raft_consensus.cc:504] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.277158 12333 leader_election.cc:288] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 683276ca765b4e6aa49d33ffea085524 (127.10.114.66:40095), 639a73312c9147aab1d84d88cd9d7293 (127.10.114.67:46033)
I20260925 09:26:39.286722 12323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:39.277782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:32915 (local address 127.10.114.67:46033)
0925 09:26:39.278014 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:39.278022 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:39.278622 (+ 600us) server_negotiation.cc:407] Connection header received
0925 09:26:39.278814 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:39.278826 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:39.278882 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:39.279032 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:39.279760 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.280847 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.284172 (+ 3325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.285512 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.285702 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:39.285778 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:39.286266 (+ 488us) server_negotiation.cc:299] Negotiation successful
0925 09:26:39.286431 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260925 09:26:39.286722 12321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:39.277609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:45913 (local address 127.10.114.66:40095)
0925 09:26:39.277839 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:39.277847 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:39.278288 (+ 441us) server_negotiation.cc:407] Connection header received
0925 09:26:39.278600 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:39.278614 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:39.278680 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:39.278817 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:39.279520 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.280727 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.284166 (+ 3439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.285476 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.285666 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:39.285771 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:39.286266 (+ 495us) server_negotiation.cc:299] Negotiation successful
0925 09:26:39.286409 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260925 09:26:39.286865 12336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:39.277453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:46033 (local address 127.10.114.65:32915)
0925 09:26:39.278520 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:39.278534 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:39.278684 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:39.279213 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:39.279217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:39.279245 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:39.279613 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:39.279621 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.280984 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.280994 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:39.283989 (+ 2995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:39.284002 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.285782 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.285786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:39.286024 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:39.286119 (+ 95us) client_negotiation.cc:769] Sending connection context
0925 09:26:39.286215 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 09:26:39.286426 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":947,"mutex_wait_us":701,"thread_start_us":361,"threads_started":1}
I20260925 09:26:39.287189 12335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:39.277453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:40095 (local address 127.10.114.65:45913)
0925 09:26:39.278162 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:39.278183 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:39.278430 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:39.278835 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:39.278841 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:39.278869 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:39.279327 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:39.279351 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.280933 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.280940 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:39.283989 (+ 3049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:39.284002 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.285764 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.285772 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:39.286028 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:39.286124 (+ 96us) client_negotiation.cc:769] Sending connection context
0925 09:26:39.286215 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 09:26:39.286426 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":278,"thread_start_us":303,"threads_started":1}
I20260925 09:26:39.287982 12262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f249c6139d3d466283a6da8451ed0698" candidate_uuid: "28001e65b61246d0b966f30e03845a70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639a73312c9147aab1d84d88cd9d7293" is_pre_election: true
I20260925 09:26:39.288074 12118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f249c6139d3d466283a6da8451ed0698" candidate_uuid: "28001e65b61246d0b966f30e03845a70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "683276ca765b4e6aa49d33ffea085524" is_pre_election: true
I20260925 09:26:39.288511 12118 raft_consensus.cc:2454] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28001e65b61246d0b966f30e03845a70 in term 0.
I20260925 09:26:39.288516 12262 raft_consensus.cc:2454] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28001e65b61246d0b966f30e03845a70 in term 0.
I20260925 09:26:39.289112 11912 leader_election.cc:302] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28001e65b61246d0b966f30e03845a70, 683276ca765b4e6aa49d33ffea085524; no voters:
I20260925 09:26:39.289508 12333 raft_consensus.cc:2781] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:26:39.289608 12333 raft_consensus.cc:482] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:26:39.289673 12333 raft_consensus.cc:3037] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:39.291716 12333 raft_consensus.cc:504] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.292349 12333 leader_election.cc:288] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [CANDIDATE]: Term 1 election: Requested vote from peers 683276ca765b4e6aa49d33ffea085524 (127.10.114.66:40095), 639a73312c9147aab1d84d88cd9d7293 (127.10.114.67:46033)
I20260925 09:26:39.292742 12118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f249c6139d3d466283a6da8451ed0698" candidate_uuid: "28001e65b61246d0b966f30e03845a70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "683276ca765b4e6aa49d33ffea085524"
I20260925 09:26:39.292780 12262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f249c6139d3d466283a6da8451ed0698" candidate_uuid: "28001e65b61246d0b966f30e03845a70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639a73312c9147aab1d84d88cd9d7293"
I20260925 09:26:39.292925 12118 raft_consensus.cc:3037] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:39.292949 12262 raft_consensus.cc:3037] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:39.294931 12262 raft_consensus.cc:2454] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28001e65b61246d0b966f30e03845a70 in term 1.
I20260925 09:26:39.295147 12118 raft_consensus.cc:2454] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28001e65b61246d0b966f30e03845a70 in term 1.
I20260925 09:26:39.295318 11910 leader_election.cc:302] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28001e65b61246d0b966f30e03845a70, 639a73312c9147aab1d84d88cd9d7293; no voters:
I20260925 09:26:39.295718 12333 raft_consensus.cc:2781] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:26:39.296612 12333 raft_consensus.cc:686] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [term 1 LEADER]: Becoming Leader. State: Replica: 28001e65b61246d0b966f30e03845a70, State: Running, Role: LEADER
I20260925 09:26:39.297290 12333 consensus_queue.cc:237] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.301753 11820 catalog_manager.cc:5495] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28001e65b61246d0b966f30e03845a70 (127.10.114.65). New cstate: current_term: 1 leader_uuid: "28001e65b61246d0b966f30e03845a70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } health_report { overall_health: UNKNOWN } } }
I20260925 09:26:39.330555 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:39.325552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60236 (local address 127.10.114.65:39319)
0925 09:26:39.325734 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:39.325738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:39.325923 (+ 185us) server_negotiation.cc:407] Connection header received
0925 09:26:39.326176 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:39.326179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:39.326274 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:39.326380 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:26:39.326749 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.327481 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.328773 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.329148 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.329235 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:39.329673 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:26:39.330107 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:26:39.330213 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:39.330319 (+ 106us) server_negotiation.cc:299] Negotiation successful
0925 09:26:39.330376 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 09:26:39.338071 12118 raft_consensus.cc:1261] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524 [term 1 FOLLOWER]: Refusing update from remote peer 28001e65b61246d0b966f30e03845a70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:26:39.338066 12262 raft_consensus.cc:1261] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293 [term 1 FOLLOWER]: Refusing update from remote peer 28001e65b61246d0b966f30e03845a70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:26:39.339088 12337 consensus_queue.cc:1035] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:26:39.339406 12333 consensus_queue.cc:1035] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:26:39.349856 12340 mvcc.cc:205] Tried to move back new op lower bound from 7333185123679666176 to 7333185123525361664. Current Snapshot: MvccSnapshot[applied={T|T < 7333185123679666176}]
I20260925 09:26:39.366170 11820 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:26:39.366472 11820 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:26:39.369861 11820 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a: Sending DeleteTablet for 3 replicas of tablet f249c6139d3d466283a6da8451ed0698
I20260925 09:26:39.371017 12237 tablet_service.cc:1514] Processing DeleteTablet for tablet f249c6139d3d466283a6da8451ed0698 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:26:39 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:32860
I20260925 09:26:39.371331 11959 tablet_service.cc:1514] Processing DeleteTablet for tablet f249c6139d3d466283a6da8451ed0698 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:26:39 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:60226
I20260925 09:26:39.372033 12098 tablet_service.cc:1514] Processing DeleteTablet for tablet f249c6139d3d466283a6da8451ed0698 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:26:39 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:45740
I20260925 09:26:39.372126 11820 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:26:39.372804 11820 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:26:39.379676 11958 tablet_service.cc:1467] Processing CreateTablet for tablet 548d3cfe341449b59350762808fa4d06 (DEFAULT_TABLE table=test-table [id=19394d6edfab47fc9a2879bb24fe0098]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:39.379946 12236 tablet_service.cc:1467] Processing CreateTablet for tablet 548d3cfe341449b59350762808fa4d06 (DEFAULT_TABLE table=test-table [id=19394d6edfab47fc9a2879bb24fe0098]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:39.380343 11958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 548d3cfe341449b59350762808fa4d06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:39.380374 12236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 548d3cfe341449b59350762808fa4d06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:39.384620 12097 tablet_service.cc:1467] Processing CreateTablet for tablet 548d3cfe341449b59350762808fa4d06 (DEFAULT_TABLE table=test-table [id=19394d6edfab47fc9a2879bb24fe0098]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:39.385013 12326 tablet_bootstrap.cc:492] T 548d3cfe341449b59350762808fa4d06 P 639a73312c9147aab1d84d88cd9d7293: Bootstrap starting.
I20260925 09:26:39.385219 12097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 548d3cfe341449b59350762808fa4d06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:39.386941 12326 tablet_bootstrap.cc:654] T 548d3cfe341449b59350762808fa4d06 P 639a73312c9147aab1d84d88cd9d7293: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:39.387395 12351 tablet_replica.cc:330] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70: stopping tablet replica
I20260925 09:26:39.388278 12351 raft_consensus.cc:2229] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:26:39.390064 12351 raft_consensus.cc:2258] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:26:39.393841 12352 tablet_replica.cc:330] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524: stopping tablet replica
I20260925 09:26:39.394287 12352 raft_consensus.cc:2229] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:26:39.394883 12352 raft_consensus.cc:2258] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:26:39.395962 12350 tablet_replica.cc:330] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293: stopping tablet replica
I20260925 09:26:39.396384 12350 raft_consensus.cc:2229] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:26:39.396457 12327 tablet_bootstrap.cc:492] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524: Bootstrap starting.
I20260925 09:26:39.396817 12350 raft_consensus.cc:2258] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:26:39.397179 12352 ts_tablet_manager.cc:1905] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:26:39.398268 12327 tablet_bootstrap.cc:654] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:39.399439 12350 ts_tablet_manager.cc:1905] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:26:39.402701 12329 tablet_bootstrap.cc:492] T 548d3cfe341449b59350762808fa4d06 P 28001e65b61246d0b966f30e03845a70: Bootstrap starting.
I20260925 09:26:39.405534 12329 tablet_bootstrap.cc:654] T 548d3cfe341449b59350762808fa4d06 P 28001e65b61246d0b966f30e03845a70: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:39.409581 12327 tablet_bootstrap.cc:492] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524: No bootstrap required, opened a new log
I20260925 09:26:39.409705 12327 ts_tablet_manager.cc:1397] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.004s
I20260925 09:26:39.410445 12327 raft_consensus.cc:348] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.410652 12327 raft_consensus.cc:374] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:39.410739 12327 raft_consensus.cc:729] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 683276ca765b4e6aa49d33ffea085524, State: Initialized, Role: FOLLOWER
I20260925 09:26:39.411021 12327 consensus_queue.cc:260] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.412456 12350 ts_tablet_manager.cc:1918] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:26:39.413918 12350 log.cc:1196] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/f249c6139d3d466283a6da8451ed0698
I20260925 09:26:39.414526 12350 ts_tablet_manager.cc:1939] T f249c6139d3d466283a6da8451ed0698 P 639a73312c9147aab1d84d88cd9d7293: Deleting consensus metadata
I20260925 09:26:39.414757 12327 ts_tablet_manager.cc:1428] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260925 09:26:39.417133 12351 ts_tablet_manager.cc:1905] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:26:39.417312 11807 catalog_manager.cc:4841] TS 639a73312c9147aab1d84d88cd9d7293 (127.10.114.67:46033): tablet f249c6139d3d466283a6da8451ed0698 (table test-table [id=c2bea88744e54ae59e871f0d26ce2e18]) successfully deleted
I20260925 09:26:39.419510 12352 ts_tablet_manager.cc:1918] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:26:39.419709 12352 log.cc:1196] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/f249c6139d3d466283a6da8451ed0698
I20260925 09:26:39.420325 12352 ts_tablet_manager.cc:1939] T f249c6139d3d466283a6da8451ed0698 P 683276ca765b4e6aa49d33ffea085524: Deleting consensus metadata
I20260925 09:26:39.422250 11809 catalog_manager.cc:4841] TS 683276ca765b4e6aa49d33ffea085524 (127.10.114.66:40095): tablet f249c6139d3d466283a6da8451ed0698 (table test-table [id=c2bea88744e54ae59e871f0d26ce2e18]) successfully deleted
I20260925 09:26:39.423820 12329 tablet_bootstrap.cc:492] T 548d3cfe341449b59350762808fa4d06 P 28001e65b61246d0b966f30e03845a70: No bootstrap required, opened a new log
I20260925 09:26:39.423960 12329 ts_tablet_manager.cc:1397] T 548d3cfe341449b59350762808fa4d06 P 28001e65b61246d0b966f30e03845a70: Time spent bootstrapping tablet: real 0.021s user 0.012s sys 0.001s
I20260925 09:26:39.424590 12351 ts_tablet_manager.cc:1918] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:26:39.424728 12329 raft_consensus.cc:348] T 548d3cfe341449b59350762808fa4d06 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.424973 12329 raft_consensus.cc:374] T 548d3cfe341449b59350762808fa4d06 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:39.425071 12329 raft_consensus.cc:729] T 548d3cfe341449b59350762808fa4d06 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28001e65b61246d0b966f30e03845a70, State: Initialized, Role: FOLLOWER
I20260925 09:26:39.425375 12329 consensus_queue.cc:260] T 548d3cfe341449b59350762808fa4d06 P 28001e65b61246d0b966f30e03845a70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.425863 12326 tablet_bootstrap.cc:492] T 548d3cfe341449b59350762808fa4d06 P 639a73312c9147aab1d84d88cd9d7293: No bootstrap required, opened a new log
I20260925 09:26:39.425988 12326 ts_tablet_manager.cc:1397] T 548d3cfe341449b59350762808fa4d06 P 639a73312c9147aab1d84d88cd9d7293: Time spent bootstrapping tablet: real 0.041s user 0.008s sys 0.013s
I20260925 09:26:39.426760 12326 raft_consensus.cc:348] T 548d3cfe341449b59350762808fa4d06 P 639a73312c9147aab1d84d88cd9d7293 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.426973 12326 raft_consensus.cc:374] T 548d3cfe341449b59350762808fa4d06 P 639a73312c9147aab1d84d88cd9d7293 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:39.427094 12326 raft_consensus.cc:729] T 548d3cfe341449b59350762808fa4d06 P 639a73312c9147aab1d84d88cd9d7293 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639a73312c9147aab1d84d88cd9d7293, State: Initialized, Role: FOLLOWER
I20260925 09:26:39.427354 12326 consensus_queue.cc:260] T 548d3cfe341449b59350762808fa4d06 P 639a73312c9147aab1d84d88cd9d7293 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.428689 12326 ts_tablet_manager.cc:1428] T 548d3cfe341449b59350762808fa4d06 P 639a73312c9147aab1d84d88cd9d7293: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260925 09:26:39.424731 12351 log.cc:1196] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/f249c6139d3d466283a6da8451ed0698
I20260925 09:26:39.429721 12351 ts_tablet_manager.cc:1939] T f249c6139d3d466283a6da8451ed0698 P 28001e65b61246d0b966f30e03845a70: Deleting consensus metadata
I20260925 09:26:39.431819 11809 catalog_manager.cc:4841] TS 28001e65b61246d0b966f30e03845a70 (127.10.114.65:39319): tablet f249c6139d3d466283a6da8451ed0698 (table test-table [id=c2bea88744e54ae59e871f0d26ce2e18]) successfully deleted
I20260925 09:26:39.428354 12329 ts_tablet_manager.cc:1428] T 548d3cfe341449b59350762808fa4d06 P 28001e65b61246d0b966f30e03845a70: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260925 09:26:39.614884 12334 raft_consensus.cc:482] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:26:39.615127 12334 raft_consensus.cc:504] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.617030 12334 leader_election.cc:288] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28001e65b61246d0b966f30e03845a70 (127.10.114.65:39319), 639a73312c9147aab1d84d88cd9d7293 (127.10.114.67:46033)
I20260925 09:26:39.624787 12323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:39.617671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:41921 (local address 127.10.114.67:46033)
0925 09:26:39.617892 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:39.617900 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:39.618875 (+ 975us) server_negotiation.cc:407] Connection header received
0925 09:26:39.619068 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:39.619073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:39.619145 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:39.619229 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:39.619676 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.620649 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.622724 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.623847 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.624008 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:39.624077 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:39.624414 (+ 337us) server_negotiation.cc:299] Negotiation successful
0925 09:26:39.624533 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 09:26:39.624851 12356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:39.617311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:46033 (local address 127.10.114.66:41921)
0925 09:26:39.618745 (+ 1434us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:39.618771 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:39.618918 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:39.619245 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:39.619249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:39.619268 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:39.619527 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:39.619535 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.620792 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.620799 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:39.622557 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:39.622575 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.624052 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.624058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:39.624235 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:39.624271 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 09:26:39.624370 (+ 99us) client_negotiation.cc:240] Negotiation successful
0925 09:26:39.624511 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1284,"thread_start_us":659,"threads_started":1}
I20260925 09:26:39.625577 12262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "548d3cfe341449b59350762808fa4d06" candidate_uuid: "683276ca765b4e6aa49d33ffea085524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639a73312c9147aab1d84d88cd9d7293" is_pre_election: true
I20260925 09:26:39.625640 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:39.617634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:56111 (local address 127.10.114.65:39319)
0925 09:26:39.617804 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:39.617811 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:39.619059 (+ 1248us) server_negotiation.cc:407] Connection header received
0925 09:26:39.619593 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:39.619604 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:39.619696 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:39.619818 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:39.620351 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.621546 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.623408 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.624421 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.624553 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:39.624623 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:39.625233 (+ 610us) server_negotiation.cc:299] Negotiation successful
0925 09:26:39.625349 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 09:26:39.626055 12357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:39.617443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:39319 (local address 127.10.114.66:56111)
0925 09:26:39.618941 (+ 1498us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:39.618955 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:39.619100 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:39.619834 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:39.619839 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:39.619863 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:39.620196 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:39.620206 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.621687 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.621693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:39.623274 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:39.623284 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.624757 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.624763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:39.625092 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:39.625116 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 09:26:39.625206 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 09:26:39.625321 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1353,"thread_start_us":1162,"threads_started":1}
I20260925 09:26:39.626101 12262 raft_consensus.cc:2454] T 548d3cfe341449b59350762808fa4d06 P 639a73312c9147aab1d84d88cd9d7293 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 683276ca765b4e6aa49d33ffea085524 in term 0.
I20260925 09:26:39.626629 12050 leader_election.cc:302] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 639a73312c9147aab1d84d88cd9d7293, 683276ca765b4e6aa49d33ffea085524; no voters:
I20260925 09:26:39.626983 12334 raft_consensus.cc:2781] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:26:39.627118 12334 raft_consensus.cc:482] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:26:39.627221 12334 raft_consensus.cc:3037] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:39.627004 11979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "548d3cfe341449b59350762808fa4d06" candidate_uuid: "683276ca765b4e6aa49d33ffea085524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28001e65b61246d0b966f30e03845a70" is_pre_election: true
I20260925 09:26:39.627462 11979 raft_consensus.cc:2454] T 548d3cfe341449b59350762808fa4d06 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 683276ca765b4e6aa49d33ffea085524 in term 0.
I20260925 09:26:39.629634 12334 raft_consensus.cc:504] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.630203 12334 leader_election.cc:288] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [CANDIDATE]: Term 1 election: Requested vote from peers 28001e65b61246d0b966f30e03845a70 (127.10.114.65:39319), 639a73312c9147aab1d84d88cd9d7293 (127.10.114.67:46033)
I20260925 09:26:39.630627 11979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "548d3cfe341449b59350762808fa4d06" candidate_uuid: "683276ca765b4e6aa49d33ffea085524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28001e65b61246d0b966f30e03845a70"
I20260925 09:26:39.630708 12262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "548d3cfe341449b59350762808fa4d06" candidate_uuid: "683276ca765b4e6aa49d33ffea085524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639a73312c9147aab1d84d88cd9d7293"
I20260925 09:26:39.630808 11979 raft_consensus.cc:3037] T 548d3cfe341449b59350762808fa4d06 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:39.630910 12262 raft_consensus.cc:3037] T 548d3cfe341449b59350762808fa4d06 P 639a73312c9147aab1d84d88cd9d7293 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:39.633443 11979 raft_consensus.cc:2454] T 548d3cfe341449b59350762808fa4d06 P 28001e65b61246d0b966f30e03845a70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 683276ca765b4e6aa49d33ffea085524 in term 1.
I20260925 09:26:39.633819 12052 leader_election.cc:302] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28001e65b61246d0b966f30e03845a70, 683276ca765b4e6aa49d33ffea085524; no voters:
I20260925 09:26:39.634068 12334 raft_consensus.cc:2781] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:26:39.634079 12262 raft_consensus.cc:2454] T 548d3cfe341449b59350762808fa4d06 P 639a73312c9147aab1d84d88cd9d7293 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 683276ca765b4e6aa49d33ffea085524 in term 1.
I20260925 09:26:39.634646 12334 raft_consensus.cc:686] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [term 1 LEADER]: Becoming Leader. State: Replica: 683276ca765b4e6aa49d33ffea085524, State: Running, Role: LEADER
I20260925 09:26:39.635177 12334 consensus_queue.cc:237] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.638474 11821 catalog_manager.cc:5495] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 reported cstate change: term changed from 0 to 1, leader changed from <none> to 683276ca765b4e6aa49d33ffea085524 (127.10.114.66). New cstate: current_term: 1 leader_uuid: "683276ca765b4e6aa49d33ffea085524" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } health_report { overall_health: UNKNOWN } } }
Sep 25 09:26:39 dist-test-slave-9pj8 krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328397, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:39.723990 12365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:39.710148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:33243 (local address 127.0.0.1:44918)
0925 09:26:39.710810 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:39.710834 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:39.711003 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:39.711380 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:39.711385 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:39.712126 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:39.712359 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:39.712368 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.713407 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.713414 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:39.714556 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:39.714567 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.715232 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.715237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:39.715453 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:39.716147 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:39.716163 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:39.718168 (+ 2005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:39.721576 (+ 3408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:39.721588 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:39.721596 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:39.721984 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:39.722316 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:39.722323 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:39.722326 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:39.722459 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:39.722945 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:39.722952 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:39.723110 (+ 158us) client_negotiation.cc:769] Sending connection context
0925 09:26:39.723279 (+ 169us) client_negotiation.cc:240] Negotiation successful
0925 09:26:39.723543 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":39,"thread_start_us":358,"threads_started":1}
I20260925 09:26:39.724012 12366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:39.710208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44918 (local address 127.10.114.126:33243)
0925 09:26:39.711108 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:39.711113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:39.711138 (+ 25us) server_negotiation.cc:407] Connection header received
0925 09:26:39.711175 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:39.711178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:39.711224 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:39.711354 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:39.712521 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.713282 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.714690 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.715088 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.715220 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:39.718353 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:39.718357 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:39.718360 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:39.718385 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:39.721439 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:39.722097 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:39.722104 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:39.722108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:39.722215 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:39.722557 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:39.722564 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:39.722568 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:39.722813 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:39.722959 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:39.723348 (+ 389us) server_negotiation.cc:299] Negotiation successful
0925 09:26:39.723647 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":141,"server-negotiator.queue_time_us":798,"thread_start_us":644,"threads_started":1}
I20260925 09:26:39.728830 11821 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:44918:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 09:26:39.729786 11821 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:26:39.736125 11958 tablet_service.cc:1467] Processing CreateTablet for tablet 108ef7f825bf4874aa65a1fcc8dd0614 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=44a5ad4d66ac4bccac19d35a904b7ebe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 09:26:39.736197 12097 tablet_service.cc:1467] Processing CreateTablet for tablet 108ef7f825bf4874aa65a1fcc8dd0614 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=44a5ad4d66ac4bccac19d35a904b7ebe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 09:26:39.736565 11958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 108ef7f825bf4874aa65a1fcc8dd0614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:39.736737 12097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 108ef7f825bf4874aa65a1fcc8dd0614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:39.740290 12237 tablet_service.cc:1467] Processing CreateTablet for tablet 108ef7f825bf4874aa65a1fcc8dd0614 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=44a5ad4d66ac4bccac19d35a904b7ebe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 09:26:39.740804 12237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 108ef7f825bf4874aa65a1fcc8dd0614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:39.742966 12329 tablet_bootstrap.cc:492] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70: Bootstrap starting.
I20260925 09:26:39.744023 12327 tablet_bootstrap.cc:492] T 108ef7f825bf4874aa65a1fcc8dd0614 P 683276ca765b4e6aa49d33ffea085524: Bootstrap starting.
I20260925 09:26:39.744742 12329 tablet_bootstrap.cc:654] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:39.746031 12327 tablet_bootstrap.cc:654] T 108ef7f825bf4874aa65a1fcc8dd0614 P 683276ca765b4e6aa49d33ffea085524: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:39.749308 12326 tablet_bootstrap.cc:492] T 108ef7f825bf4874aa65a1fcc8dd0614 P 639a73312c9147aab1d84d88cd9d7293: Bootstrap starting.
I20260925 09:26:39.751366 12326 tablet_bootstrap.cc:654] T 108ef7f825bf4874aa65a1fcc8dd0614 P 639a73312c9147aab1d84d88cd9d7293: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:39.754837 12329 tablet_bootstrap.cc:492] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70: No bootstrap required, opened a new log
I20260925 09:26:39.755035 12329 ts_tablet_manager.cc:1397] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260925 09:26:39.755640 12326 tablet_bootstrap.cc:492] T 108ef7f825bf4874aa65a1fcc8dd0614 P 639a73312c9147aab1d84d88cd9d7293: No bootstrap required, opened a new log
I20260925 09:26:39.755796 12327 tablet_bootstrap.cc:492] T 108ef7f825bf4874aa65a1fcc8dd0614 P 683276ca765b4e6aa49d33ffea085524: No bootstrap required, opened a new log
I20260925 09:26:39.755762 12326 ts_tablet_manager.cc:1397] T 108ef7f825bf4874aa65a1fcc8dd0614 P 639a73312c9147aab1d84d88cd9d7293: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260925 09:26:39.755961 12327 ts_tablet_manager.cc:1397] T 108ef7f825bf4874aa65a1fcc8dd0614 P 683276ca765b4e6aa49d33ffea085524: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260925 09:26:39.755874 12329 raft_consensus.cc:348] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.756145 12329 raft_consensus.cc:374] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:39.756218 12329 raft_consensus.cc:729] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28001e65b61246d0b966f30e03845a70, State: Initialized, Role: FOLLOWER
I20260925 09:26:39.756476 12329 consensus_queue.cc:260] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.756580 12326 raft_consensus.cc:348] T 108ef7f825bf4874aa65a1fcc8dd0614 P 639a73312c9147aab1d84d88cd9d7293 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.756654 12327 raft_consensus.cc:348] T 108ef7f825bf4874aa65a1fcc8dd0614 P 683276ca765b4e6aa49d33ffea085524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.756794 12326 raft_consensus.cc:374] T 108ef7f825bf4874aa65a1fcc8dd0614 P 639a73312c9147aab1d84d88cd9d7293 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:39.756861 12326 raft_consensus.cc:729] T 108ef7f825bf4874aa65a1fcc8dd0614 P 639a73312c9147aab1d84d88cd9d7293 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639a73312c9147aab1d84d88cd9d7293, State: Initialized, Role: FOLLOWER
I20260925 09:26:39.756870 12327 raft_consensus.cc:374] T 108ef7f825bf4874aa65a1fcc8dd0614 P 683276ca765b4e6aa49d33ffea085524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:39.756943 12327 raft_consensus.cc:729] T 108ef7f825bf4874aa65a1fcc8dd0614 P 683276ca765b4e6aa49d33ffea085524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 683276ca765b4e6aa49d33ffea085524, State: Initialized, Role: FOLLOWER
I20260925 09:26:39.756908 12345 tablet_replica.cc:439] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } } }
I20260925 09:26:39.757076 12326 consensus_queue.cc:260] T 108ef7f825bf4874aa65a1fcc8dd0614 P 639a73312c9147aab1d84d88cd9d7293 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.757277 12345 tablet_replica.cc:442] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:26:39.757164 12327 consensus_queue.cc:260] T 108ef7f825bf4874aa65a1fcc8dd0614 P 683276ca765b4e6aa49d33ffea085524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.757540 12326 ts_tablet_manager.cc:1428] T 108ef7f825bf4874aa65a1fcc8dd0614 P 639a73312c9147aab1d84d88cd9d7293: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260925 09:26:39.757632 12327 ts_tablet_manager.cc:1428] T 108ef7f825bf4874aa65a1fcc8dd0614 P 683276ca765b4e6aa49d33ffea085524: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260925 09:26:39.757995 12334 tablet_replica.cc:439] T 108ef7f825bf4874aa65a1fcc8dd0614 P 683276ca765b4e6aa49d33ffea085524: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } } }
I20260925 09:26:39.758275 12334 tablet_replica.cc:442] T 108ef7f825bf4874aa65a1fcc8dd0614 P 683276ca765b4e6aa49d33ffea085524: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:26:39.759352 12329 ts_tablet_manager.cc:1428] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260925 09:26:39.759742 12332 tablet_replica.cc:439] T 108ef7f825bf4874aa65a1fcc8dd0614 P 639a73312c9147aab1d84d88cd9d7293: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } } }
I20260925 09:26:39.760053 12332 tablet_replica.cc:442] T 108ef7f825bf4874aa65a1fcc8dd0614 P 639a73312c9147aab1d84d88cd9d7293: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:26:39.943356 12345 raft_consensus.cc:482] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:26:39.943552 12345 raft_consensus.cc:504] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.944231 12345 leader_election.cc:288] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 683276ca765b4e6aa49d33ffea085524 (127.10.114.66:40095), 639a73312c9147aab1d84d88cd9d7293 (127.10.114.67:46033)
I20260925 09:26:39.944689 12118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "108ef7f825bf4874aa65a1fcc8dd0614" candidate_uuid: "28001e65b61246d0b966f30e03845a70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "683276ca765b4e6aa49d33ffea085524" is_pre_election: true
I20260925 09:26:39.944733 12262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "108ef7f825bf4874aa65a1fcc8dd0614" candidate_uuid: "28001e65b61246d0b966f30e03845a70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639a73312c9147aab1d84d88cd9d7293" is_pre_election: true
I20260925 09:26:39.944916 12118 raft_consensus.cc:2454] T 108ef7f825bf4874aa65a1fcc8dd0614 P 683276ca765b4e6aa49d33ffea085524 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28001e65b61246d0b966f30e03845a70 in term 0.
I20260925 09:26:39.945010 12262 raft_consensus.cc:2454] T 108ef7f825bf4874aa65a1fcc8dd0614 P 639a73312c9147aab1d84d88cd9d7293 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28001e65b61246d0b966f30e03845a70 in term 0.
I20260925 09:26:39.945303 11912 leader_election.cc:302] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28001e65b61246d0b966f30e03845a70, 683276ca765b4e6aa49d33ffea085524; no voters:
I20260925 09:26:39.945572 12345 raft_consensus.cc:2781] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:26:39.945660 12345 raft_consensus.cc:482] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:26:39.945716 12345 raft_consensus.cc:3037] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:39.947579 12345 raft_consensus.cc:504] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.948103 12345 leader_election.cc:288] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70 [CANDIDATE]: Term 1 election: Requested vote from peers 683276ca765b4e6aa49d33ffea085524 (127.10.114.66:40095), 639a73312c9147aab1d84d88cd9d7293 (127.10.114.67:46033)
I20260925 09:26:39.948470 12262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "108ef7f825bf4874aa65a1fcc8dd0614" candidate_uuid: "28001e65b61246d0b966f30e03845a70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639a73312c9147aab1d84d88cd9d7293"
I20260925 09:26:39.948495 12118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "108ef7f825bf4874aa65a1fcc8dd0614" candidate_uuid: "28001e65b61246d0b966f30e03845a70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "683276ca765b4e6aa49d33ffea085524"
I20260925 09:26:39.948666 12262 raft_consensus.cc:3037] T 108ef7f825bf4874aa65a1fcc8dd0614 P 639a73312c9147aab1d84d88cd9d7293 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:39.948663 12118 raft_consensus.cc:3037] T 108ef7f825bf4874aa65a1fcc8dd0614 P 683276ca765b4e6aa49d33ffea085524 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:39.950515 12118 raft_consensus.cc:2454] T 108ef7f825bf4874aa65a1fcc8dd0614 P 683276ca765b4e6aa49d33ffea085524 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28001e65b61246d0b966f30e03845a70 in term 1.
I20260925 09:26:39.950592 12262 raft_consensus.cc:2454] T 108ef7f825bf4874aa65a1fcc8dd0614 P 639a73312c9147aab1d84d88cd9d7293 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28001e65b61246d0b966f30e03845a70 in term 1.
I20260925 09:26:39.950924 11912 leader_election.cc:302] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28001e65b61246d0b966f30e03845a70, 683276ca765b4e6aa49d33ffea085524; no voters:
I20260925 09:26:39.951241 12345 raft_consensus.cc:2781] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:26:39.951658 12345 raft_consensus.cc:686] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70 [term 1 LEADER]: Becoming Leader. State: Replica: 28001e65b61246d0b966f30e03845a70, State: Running, Role: LEADER
I20260925 09:26:39.951954 12345 consensus_queue.cc:237] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } }
I20260925 09:26:39.953477 12370 tablet_replica.cc:439] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70: TxnStatusTablet state changed. Reason: New leader 28001e65b61246d0b966f30e03845a70. Latest consensus state: current_term: 1 leader_uuid: "28001e65b61246d0b966f30e03845a70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } } }
I20260925 09:26:39.953702 12370 tablet_replica.cc:442] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70: This TxnStatusTablet replica's current role is: LEADER
I20260925 09:26:39.954279 12372 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 09:26:39.954499 12372 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 09:26:39.955011 11822 catalog_manager.cc:5495] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28001e65b61246d0b966f30e03845a70 (127.10.114.65). New cstate: current_term: 1 leader_uuid: "28001e65b61246d0b966f30e03845a70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } health_report { overall_health: UNKNOWN } } }
I20260925 09:26:40.003827 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:39.997166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60246 (local address 127.10.114.65:39319)
0925 09:26:39.997348 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:39.997354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:39.997570 (+ 216us) server_negotiation.cc:407] Connection header received
0925 09:26:39.997825 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:39.997831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:39.997907 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:39.997999 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:26:39.998424 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.999614 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.001411 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.002431 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.002552 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:40.003079 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:26:40.003204 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:26:40.003285 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:40.003449 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 09:26:40.003526 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 09:26:40.003916 12365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:39.997166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:39319 (local address 127.0.0.1:60246)
0925 09:26:39.997477 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:39.997496 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:39.997622 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:39.998052 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:39.998057 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:39.998070 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:26:39.998277 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:39.998288 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:39.999806 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:39.999812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:40.001274 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:40.001287 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.002589 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.002595 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:40.002813 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:40.002901 (+ 88us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:26:40.003306 (+ 405us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:26:40.003331 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 09:26:40.003417 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 09:26:40.003565 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179}
I20260925 09:26:40.010021 12262 raft_consensus.cc:1261] T 108ef7f825bf4874aa65a1fcc8dd0614 P 639a73312c9147aab1d84d88cd9d7293 [term 1 FOLLOWER]: Refusing update from remote peer 28001e65b61246d0b966f30e03845a70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:26:40.010095 12118 raft_consensus.cc:1261] T 108ef7f825bf4874aa65a1fcc8dd0614 P 683276ca765b4e6aa49d33ffea085524 [term 1 FOLLOWER]: Refusing update from remote peer 28001e65b61246d0b966f30e03845a70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:26:40.010646 12370 consensus_queue.cc:1035] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:26:40.010996 12345 consensus_queue.cc:1035] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:26:40.028291 12334 tablet_replica.cc:439] T 108ef7f825bf4874aa65a1fcc8dd0614 P 683276ca765b4e6aa49d33ffea085524: TxnStatusTablet state changed. Reason: New leader 28001e65b61246d0b966f30e03845a70. Latest consensus state: current_term: 1 leader_uuid: "28001e65b61246d0b966f30e03845a70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } } }
I20260925 09:26:40.028576 12334 tablet_replica.cc:442] T 108ef7f825bf4874aa65a1fcc8dd0614 P 683276ca765b4e6aa49d33ffea085524: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:26:40.032398 12334 tablet_replica.cc:439] T 108ef7f825bf4874aa65a1fcc8dd0614 P 683276ca765b4e6aa49d33ffea085524: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "28001e65b61246d0b966f30e03845a70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } } }
I20260925 09:26:40.032761 12334 tablet_replica.cc:442] T 108ef7f825bf4874aa65a1fcc8dd0614 P 683276ca765b4e6aa49d33ffea085524: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:26:40.032981 12376 mvcc.cc:205] Tried to move back new op lower bound from 7333185126435446784 to 7333185126205018112. Current Snapshot: MvccSnapshot[applied={T|T < 7333185126435446784}]
I20260925 09:26:40.035573 12345 tablet_replica.cc:439] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28001e65b61246d0b966f30e03845a70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } } }
I20260925 09:26:40.035959 12345 tablet_replica.cc:442] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70: This TxnStatusTablet replica's current role is: LEADER
I20260925 09:26:40.037122 12332 tablet_replica.cc:439] T 108ef7f825bf4874aa65a1fcc8dd0614 P 639a73312c9147aab1d84d88cd9d7293: TxnStatusTablet state changed. Reason: New leader 28001e65b61246d0b966f30e03845a70. Latest consensus state: current_term: 1 leader_uuid: "28001e65b61246d0b966f30e03845a70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } } }
I20260925 09:26:40.037371 12332 tablet_replica.cc:442] T 108ef7f825bf4874aa65a1fcc8dd0614 P 639a73312c9147aab1d84d88cd9d7293: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:26:40.040270 12382 tablet_replica.cc:439] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28001e65b61246d0b966f30e03845a70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } } }
I20260925 09:26:40.040737 12382 tablet_replica.cc:442] T 108ef7f825bf4874aa65a1fcc8dd0614 P 28001e65b61246d0b966f30e03845a70: This TxnStatusTablet replica's current role is: LEADER
I20260925 09:26:40.044811 12332 tablet_replica.cc:439] T 108ef7f825bf4874aa65a1fcc8dd0614 P 639a73312c9147aab1d84d88cd9d7293: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "28001e65b61246d0b966f30e03845a70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 } } peers { permanent_uuid: "683276ca765b4e6aa49d33ffea085524" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40095 } } peers { permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 } } }
I20260925 09:26:40.045033 12332 tablet_replica.cc:442] T 108ef7f825bf4874aa65a1fcc8dd0614 P 639a73312c9147aab1d84d88cd9d7293: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:26:40.052520 12386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:40.044956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45752 (local address 127.10.114.66:40095)
0925 09:26:40.046600 (+ 1644us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:40.046610 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:40.047154 (+ 544us) server_negotiation.cc:407] Connection header received
0925 09:26:40.047362 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:40.047366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:40.047433 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:40.047583 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:26:40.047991 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.048827 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.050220 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.050720 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.050843 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:40.051276 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:26:40.051907 (+ 631us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:26:40.052059 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:40.052134 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 09:26:40.052204 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1470,"thread_start_us":1350,"threads_started":1}
I20260925 09:26:40.068801 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:40.062261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:39319 (local address 127.10.114.66:57091)
0925 09:26:40.062841 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:40.062878 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:40.063026 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:40.063363 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:40.063365 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:40.063374 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:26:40.063662 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:40.063670 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.065090 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.065093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:40.066674 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:40.066686 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.067696 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.067701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:40.067881 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:40.067940 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:26:40.068343 (+ 403us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:26:40.068351 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 09:26:40.068441 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 09:26:40.068532 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":350,"threads_started":1}
I20260925 09:26:40.068799 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:40.062273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:57091 (local address 127.10.114.65:39319)
0925 09:26:40.062533 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:40.062541 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:40.063006 (+ 465us) server_negotiation.cc:407] Connection header received
0925 09:26:40.063162 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:40.063168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:40.063258 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:40.063345 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:26:40.063826 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.064974 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.066805 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.067511 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.067625 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:40.068102 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:26:40.068238 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:26:40.068322 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:40.068462 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 09:26:40.068532 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:26:40.078302 12262 raft_consensus.cc:1261] T 548d3cfe341449b59350762808fa4d06 P 639a73312c9147aab1d84d88cd9d7293 [term 1 FOLLOWER]: Refusing update from remote peer 683276ca765b4e6aa49d33ffea085524: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:26:40.078699 11979 raft_consensus.cc:1261] T 548d3cfe341449b59350762808fa4d06 P 28001e65b61246d0b966f30e03845a70 [term 1 FOLLOWER]: Refusing update from remote peer 683276ca765b4e6aa49d33ffea085524: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:26:40.078959 12358 consensus_queue.cc:1035] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639a73312c9147aab1d84d88cd9d7293" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:26:40.079496 12334 consensus_queue.cc:1035] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28001e65b61246d0b966f30e03845a70" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:26:40.113029 12386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:40.105748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:53727 (local address 127.10.114.66:40095)
0925 09:26:40.105919 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:40.105926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:40.106578 (+ 652us) server_negotiation.cc:407] Connection header received
0925 09:26:40.106811 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:40.106816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:40.106892 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:40.106990 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:26:40.107728 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.108848 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.110481 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.111122 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.111240 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:40.112186 (+ 946us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:26:40.112296 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:26:40.112379 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:40.112655 (+ 276us) server_negotiation.cc:299] Negotiation successful
0925 09:26:40.112733 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 09:26:40.113068 12393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:40.105822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:40095 (local address 127.10.114.65:53727)
0925 09:26:40.106468 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:40.106506 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:40.106657 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:40.107042 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:40.107046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:40.107058 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:26:40.107514 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:40.107528 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.108997 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.109003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:40.110340 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:40.110352 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.111285 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.111291 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:40.111901 (+ 610us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:40.111977 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:26:40.112429 (+ 452us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:26:40.112439 (+ 10us) client_negotiation.cc:769] Sending connection context
0925 09:26:40.112518 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 09:26:40.112646 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":314,"threads_started":1}
I20260925 09:26:40.159735 11820 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:26:40.160008 11820 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:26:40.163215 11820 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 993396ef0d824de1afaaa25ef205204a: Sending DeleteTablet for 3 replicas of tablet 548d3cfe341449b59350762808fa4d06
I20260925 09:26:40.164002 12237 tablet_service.cc:1514] Processing DeleteTablet for tablet 548d3cfe341449b59350762808fa4d06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:26:40 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:32860
I20260925 09:26:40.164348 11958 tablet_service.cc:1514] Processing DeleteTablet for tablet 548d3cfe341449b59350762808fa4d06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:26:40 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:60226
I20260925 09:26:40.164976 12097 tablet_service.cc:1514] Processing DeleteTablet for tablet 548d3cfe341449b59350762808fa4d06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:26:40 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:45740
I20260925 09:26:40.187709 12397 tablet_replica.cc:330] T 548d3cfe341449b59350762808fa4d06 P 28001e65b61246d0b966f30e03845a70: stopping tablet replica
I20260925 09:26:40.188190 12398 tablet_replica.cc:330] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524: stopping tablet replica
I20260925 09:26:40.188519 12398 raft_consensus.cc:2229] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:26:40.188867 12397 raft_consensus.cc:2229] T 548d3cfe341449b59350762808fa4d06 P 28001e65b61246d0b966f30e03845a70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:26:40.189507 12398 raft_consensus.cc:2258] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:26:40.189653 12396 tablet_replica.cc:330] T 548d3cfe341449b59350762808fa4d06 P 639a73312c9147aab1d84d88cd9d7293: stopping tablet replica
I20260925 09:26:40.189839 12396 raft_consensus.cc:2229] T 548d3cfe341449b59350762808fa4d06 P 639a73312c9147aab1d84d88cd9d7293 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:26:40.189956 12397 raft_consensus.cc:2258] T 548d3cfe341449b59350762808fa4d06 P 28001e65b61246d0b966f30e03845a70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:26:40.190057 12396 raft_consensus.cc:2258] T 548d3cfe341449b59350762808fa4d06 P 639a73312c9147aab1d84d88cd9d7293 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:26:40.191169 12396 ts_tablet_manager.cc:1905] T 548d3cfe341449b59350762808fa4d06 P 639a73312c9147aab1d84d88cd9d7293: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:26:40.191182 12397 ts_tablet_manager.cc:1905] T 548d3cfe341449b59350762808fa4d06 P 28001e65b61246d0b966f30e03845a70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:26:40.196868 12398 ts_tablet_manager.cc:1905] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:26:40.203186 12396 ts_tablet_manager.cc:1918] T 548d3cfe341449b59350762808fa4d06 P 639a73312c9147aab1d84d88cd9d7293: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:26:40.207232 12398 ts_tablet_manager.cc:1918] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:26:40.208089 12398 log.cc:1196] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/548d3cfe341449b59350762808fa4d06
I20260925 09:26:40.208527 12397 ts_tablet_manager.cc:1918] T 548d3cfe341449b59350762808fa4d06 P 28001e65b61246d0b966f30e03845a70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:26:40.208968 12398 ts_tablet_manager.cc:1939] T 548d3cfe341449b59350762808fa4d06 P 683276ca765b4e6aa49d33ffea085524: Deleting consensus metadata
I20260925 09:26:40.209134 12397 log.cc:1196] T 548d3cfe341449b59350762808fa4d06 P 28001e65b61246d0b966f30e03845a70: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/548d3cfe341449b59350762808fa4d06
I20260925 09:26:40.209687 12397 ts_tablet_manager.cc:1939] T 548d3cfe341449b59350762808fa4d06 P 28001e65b61246d0b966f30e03845a70: Deleting consensus metadata
Sep 25 09:26:40 dist-test-slave-9pj8 krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328398, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:26:40.210359 12396 log.cc:1196] T 548d3cfe341449b59350762808fa4d06 P 639a73312c9147aab1d84d88cd9d7293: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/548d3cfe341449b59350762808fa4d06
I20260925 09:26:40.210822 12396 ts_tablet_manager.cc:1939] T 548d3cfe341449b59350762808fa4d06 P 639a73312c9147aab1d84d88cd9d7293: Deleting consensus metadata
I20260925 09:26:40.210882 11809 catalog_manager.cc:4841] TS 683276ca765b4e6aa49d33ffea085524 (127.10.114.66:40095): tablet 548d3cfe341449b59350762808fa4d06 (table test-table [id=19394d6edfab47fc9a2879bb24fe0098]) successfully deleted
I20260925 09:26:40.211838 11807 catalog_manager.cc:4841] TS 639a73312c9147aab1d84d88cd9d7293 (127.10.114.67:46033): tablet 548d3cfe341449b59350762808fa4d06 (table test-table [id=19394d6edfab47fc9a2879bb24fe0098]) successfully deleted
I20260925 09:26:40.212229 11809 catalog_manager.cc:4841] TS 28001e65b61246d0b966f30e03845a70 (127.10.114.65:39319): tablet 548d3cfe341449b59350762808fa4d06 (table test-table [id=19394d6edfab47fc9a2879bb24fe0098]) successfully deleted
I20260925 09:26:40.220388 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:40.186783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60260 (local address 127.10.114.65:39319)
0925 09:26:40.186974 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:40.186980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:40.194275 (+ 7295us) server_negotiation.cc:407] Connection header received
0925 09:26:40.194725 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:40.194732 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:40.194814 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:40.194911 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:40.198137 (+ 3226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.199296 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.205115 (+ 5819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.205849 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.205956 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:40.213179 (+ 7223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:40.213193 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:40.213213 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:40.213268 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:40.217106 (+ 3838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:40.217909 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:40.217919 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:40.217922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:40.218064 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:40.218559 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:40.218565 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:40.218569 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:40.219033 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:40.219369 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:40.219654 (+ 285us) server_negotiation.cc:299] Negotiation successful
0925 09:26:40.219886 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260925 09:26:40.221047 12024 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60260
I20260925 09:26:40.243542 12402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:40.224528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44926 (local address 127.10.114.126:33243)
0925 09:26:40.225010 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:40.225016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:40.228236 (+ 3220us) server_negotiation.cc:407] Connection header received
0925 09:26:40.228388 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:40.228395 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:40.228480 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:40.228589 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:40.230191 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.231437 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.233612 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.234280 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.234406 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:40.236968 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:40.236973 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:40.236978 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:40.237013 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:40.240744 (+ 3731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:40.241506 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:40.241512 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:40.241516 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:40.241656 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:40.242055 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:40.242061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:40.242065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:40.242383 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:40.242648 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:40.242947 (+ 299us) server_negotiation.cc:299] Negotiation successful
0925 09:26:40.243161 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":199,"threads_started":1}
W20260925 09:26:40.244088 11820 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44926
I20260925 09:26:40.246651 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 11893
I20260925 09:26:40.265367 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:40.248239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60276 (local address 127.10.114.65:39319)
0925 09:26:40.248403 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:40.248409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:40.248767 (+ 358us) server_negotiation.cc:407] Connection header received
0925 09:26:40.249013 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:40.249018 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:40.249102 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:40.249173 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:40.250785 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.252014 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.254193 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.254832 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.254925 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:40.257754 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:40.257760 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:40.257764 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:40.257799 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:40.261470 (+ 3671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:40.262553 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:40.262559 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:40.262575 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:40.262731 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:40.263623 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:40.263628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:40.263632 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:40.264027 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:40.264303 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:40.264576 (+ 273us) server_negotiation.cc:299] Negotiation successful
0925 09:26:40.264802 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260925 09:26:40.266018 12024 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60276
W20260925 09:26:40.266615 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 11893: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:26:40.266786 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 11893
I20260925 09:26:40.290277 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 12033
Sep 25 09:26:40 dist-test-slave-9pj8 krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328398, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.114.66@KRBTEST.COM
I20260925 09:26:40.307801 12386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:40.290986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45758 (local address 127.10.114.66:40095)
0925 09:26:40.291156 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:40.291167 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:40.291299 (+ 132us) server_negotiation.cc:407] Connection header received
0925 09:26:40.291548 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:40.291551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:40.291629 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:40.291719 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:40.293262 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.294012 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.295935 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.296374 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.296462 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:40.301073 (+ 4611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:40.301088 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:40.301107 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:40.301152 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:40.304992 (+ 3840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:40.305678 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:40.305708 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:40.305711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:40.305868 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:40.306195 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:40.306200 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:40.306203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:40.306653 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:40.306966 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:40.307183 (+ 217us) server_negotiation.cc:299] Negotiation successful
0925 09:26:40.307394 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260925 09:26:40.308498 12163 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45758
W20260925 09:26:40.309082 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 12033: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:26:40.309231 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 12033
I20260925 09:26:40.324015 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 12172
Sep 25 09:26:40 dist-test-slave-9pj8 krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328398, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.114.67@KRBTEST.COM
I20260925 09:26:40.337673 12405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:40.324833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32866 (local address 127.10.114.67:46033)
0925 09:26:40.325275 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:40.325279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:40.325293 (+ 14us) server_negotiation.cc:407] Connection header received
0925 09:26:40.325340 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:40.325343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:40.325401 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:40.325669 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:40.326818 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.327483 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.328774 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:40.329191 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:40.329285 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:40.333088 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:40.333099 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:40.333114 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:40.333140 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:40.335496 (+ 2356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:40.336093 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:40.336100 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:40.336102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:40.336209 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:40.336491 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:40.336494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:40.336496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:40.336779 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:40.336968 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:40.337174 (+ 206us) server_negotiation.cc:299] Negotiation successful
0925 09:26:40.337316 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":250,"threads_started":1}
W20260925 09:26:40.338294 12302 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32866
W20260925 09:26:40.338918 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 12172: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:26:40.339054 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 12172
I20260925 09:26:40.355225 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 11790
I20260925 09:26:40.356221 11862 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:26:40.488029 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 11790
2026-09-25T09:26:40Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4839 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:26:40 dist-test-slave-9pj8 krb5kdc[12409](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:26:40 dist-test-slave-9pj8 krb5kdc[12409](info): set up 2 sockets
Sep 25 09:26:40 dist-test-slave-9pj8 krb5kdc[12409](info): commencing operation
krb5kdc: starting...
W20260925 09:26:41.544426 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.004s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:26:41 dist-test-slave-9pj8 krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:26:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:26:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:26:41.723064 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:42763
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:40449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:42763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:42.024210 12422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:42.024492 12422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:42.024678 12422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:42.031616 12422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:26:42.031702 12422 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 09:26:42.031800 12422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:42.031888 12422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:42.031965 12422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:26:42.032066 12422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:26:42.041422 12422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:40449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:42763
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:42763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:42.042640 12422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:42.044039 12422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:42.055280 12427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:42.055306 12428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:42.055341 12430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:42.056465 12422 server_base.cc:1034] running on GCE node
I20260925 09:26:42.057255 12422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:42.058498 12422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:42.059748 12422 hybrid_clock.cc:648] HybridClock initialized: now 1790328402059684 us; error 46 us; skew 500 ppm
I20260925 09:26:42.060092 12422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:42 dist-test-slave-9pj8 krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328402, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:42.063017 12422 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:26:42.064278 12422 webserver.cc:460] Webserver started at http://127.10.114.126:36183/ using document root <none> and password file <none>
I20260925 09:26:42.064864 12422 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:42.064939 12422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:42.065166 12422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:42.067873 12422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "8472853bdebe4d7f99bfaddcd21d4dbf"
format_stamp: "Formatted at 2026-09-25 09:26:42 on dist-test-slave-9pj8"
server_key: "fa5147198ba57ffb63eb691517aa4404"
server_key_iv: "1ebcc6b0a7e4f99a720b07f6536a76ad"
server_key_version: "encryptionkey@0"
I20260925 09:26:42.068504 12422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "8472853bdebe4d7f99bfaddcd21d4dbf"
format_stamp: "Formatted at 2026-09-25 09:26:42 on dist-test-slave-9pj8"
server_key: "fa5147198ba57ffb63eb691517aa4404"
server_key_iv: "1ebcc6b0a7e4f99a720b07f6536a76ad"
server_key_version: "encryptionkey@0"
I20260925 09:26:42.073154 12422 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:26:42.076180 12437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:42.077399 12422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:26:42.077518 12422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "8472853bdebe4d7f99bfaddcd21d4dbf"
format_stamp: "Formatted at 2026-09-25 09:26:42 on dist-test-slave-9pj8"
server_key: "fa5147198ba57ffb63eb691517aa4404"
server_key_iv: "1ebcc6b0a7e4f99a720b07f6536a76ad"
server_key_version: "encryptionkey@0"
I20260925 09:26:42.077687 12422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:42.102185 12422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:42.106233 12422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:42.106444 12422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:42.127446 12422 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:42763
I20260925 09:26:42.127544 12498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:42763 every 8 connection(s)
I20260925 09:26:42.128619 12422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:26:42.132294 12499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:42.137509 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 12422
I20260925 09:26:42.137712 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:26:42.138543 10697 external_mini_cluster.cc:1428] Setting key d07b6d33a18f55d149c1433f3d806e2e
I20260925 09:26:42.142866 12499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf: Bootstrap starting.
I20260925 09:26:42.146286 12499 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:42.147481 12499 log.cc:824] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf: Log is configured to *not* fsync() on all Append() calls
I20260925 09:26:42.150429 12499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf: No bootstrap required, opened a new log
Sep 25 09:26:42 dist-test-slave-9pj8 krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:42.158370 12499 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8472853bdebe4d7f99bfaddcd21d4dbf" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42763 } }
I20260925 09:26:42.158712 12499 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:42.158844 12499 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8472853bdebe4d7f99bfaddcd21d4dbf, State: Initialized, Role: FOLLOWER
I20260925 09:26:42.159473 12499 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8472853bdebe4d7f99bfaddcd21d4dbf" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42763 } }
I20260925 09:26:42.159726 12499 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:26:42.159891 12499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:26:42.160074 12499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:42.160434 12502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:42.140570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47662 (local address 127.10.114.126:42763)
0925 09:26:42.141326 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:42.141337 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:42.141372 (+ 35us) server_negotiation.cc:407] Connection header received
0925 09:26:42.142896 (+ 1524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:42.142934 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:42.143330 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:42.143614 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:42.145294 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:42.146519 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:42.148308 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:42.149114 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:42.149238 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:42.153577 (+ 4339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:42.153584 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:42.153596 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:42.153626 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:42.156576 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:42.157183 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:42.157190 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:42.157198 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:42.157338 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:42.157724 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:42.157728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:42.157730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:42.158172 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:42.158478 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:42.158865 (+ 387us) server_negotiation.cc:299] Negotiation successful
0925 09:26:42.159300 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":364,"threads_started":1}
I20260925 09:26:42.162194 12499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8472853bdebe4d7f99bfaddcd21d4dbf" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42763 } }
I20260925 09:26:42.162675 12499 leader_election.cc:302] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8472853bdebe4d7f99bfaddcd21d4dbf; no voters:
I20260925 09:26:42.163255 12499 leader_election.cc:288] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:26:42.163414 12504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:26:42.164124 12504 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf [term 1 LEADER]: Becoming Leader. State: Replica: 8472853bdebe4d7f99bfaddcd21d4dbf, State: Running, Role: LEADER
I20260925 09:26:42.164572 12504 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8472853bdebe4d7f99bfaddcd21d4dbf" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42763 } }
I20260925 09:26:42.165057 12499 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:26:42.167037 12505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8472853bdebe4d7f99bfaddcd21d4dbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8472853bdebe4d7f99bfaddcd21d4dbf" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42763 } } }
I20260925 09:26:42.167716 12505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf [sys.catalog]: This master's current role is: LEADER
I20260925 09:26:42.167042 12506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8472853bdebe4d7f99bfaddcd21d4dbf. Latest consensus state: current_term: 1 leader_uuid: "8472853bdebe4d7f99bfaddcd21d4dbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8472853bdebe4d7f99bfaddcd21d4dbf" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42763 } } }
I20260925 09:26:42.168573 12506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf [sys.catalog]: This master's current role is: LEADER
I20260925 09:26:42.172135 12509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:26:42.177471 12509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:26:42.185047 12509 catalog_manager.cc:1347] Generated new cluster ID: 8c2d0ba2511b4d7abf62c28114911182
I20260925 09:26:42.185133 12509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:26:42.206915 12509 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:26:42.208333 12509 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:26:42.216289 12509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf: Generated new TSK 0
I20260925 09:26:42.217018 12509 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
I20260925 09:26:42.273535 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42763
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:40449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:42.638597 12525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:42.638850 12525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:42.638952 12525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:42.645781 12525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:42.645872 12525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:42.645952 12525 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 09:26:42.646025 12525 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 09:26:42.646102 12525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:26:42.655141 12525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:40449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.114.126:42763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:42.656227 12525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:42.657464 12525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:42.670593 12531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:42.670783 12530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:42.671614 12533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:42.671990 12525 server_base.cc:1034] running on GCE node
I20260925 09:26:42.672574 12525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:42.673573 12525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:42.674777 12525 hybrid_clock.cc:648] HybridClock initialized: now 1790328402674720 us; error 42 us; skew 500 ppm
I20260925 09:26:42.675109 12525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:42 dist-test-slave-9pj8 krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328402, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:42.678153 12525 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:26:42.679589 12525 webserver.cc:460] Webserver started at http://127.10.114.65:39949/ using document root <none> and password file <none>
I20260925 09:26:42.680238 12525 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:42.680335 12525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:42.680594 12525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:42.683272 12525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "146f2508850b4ffd81f52247de045025"
format_stamp: "Formatted at 2026-09-25 09:26:42 on dist-test-slave-9pj8"
server_key: "cd0b45cbfc2963eca4a8212edbd80000"
server_key_iv: "c5f1ad51c391cdf1e48818559a9ba5a2"
server_key_version: "encryptionkey@0"
I20260925 09:26:42.683961 12525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "146f2508850b4ffd81f52247de045025"
format_stamp: "Formatted at 2026-09-25 09:26:42 on dist-test-slave-9pj8"
server_key: "cd0b45cbfc2963eca4a8212edbd80000"
server_key_iv: "c5f1ad51c391cdf1e48818559a9ba5a2"
server_key_version: "encryptionkey@0"
I20260925 09:26:42.688337 12525 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:26:42.691367 12540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:42.692692 12525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:26:42.692865 12525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "146f2508850b4ffd81f52247de045025"
format_stamp: "Formatted at 2026-09-25 09:26:42 on dist-test-slave-9pj8"
server_key: "cd0b45cbfc2963eca4a8212edbd80000"
server_key_iv: "c5f1ad51c391cdf1e48818559a9ba5a2"
server_key_version: "encryptionkey@0"
I20260925 09:26:42.693058 12525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:42.722242 12525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:42.725950 12525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:42.726214 12525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:42.728703 12525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:26:42.728806 12525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:42.728943 12525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:26:42.729069 12525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 09:26:42 dist-test-slave-9pj8 krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328402, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:42.752192 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:42.729140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:37125 (local address 127.10.114.126:42763)
0925 09:26:42.730162 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:42.730170 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:42.730729 (+ 559us) server_negotiation.cc:407] Connection header received
0925 09:26:42.733024 (+ 2295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:42.733030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:42.733135 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:42.733257 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:42.735417 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:42.736673 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:42.739201 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:42.739961 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:42.740123 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:42.745027 (+ 4904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:42.745047 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:42.745055 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:42.745104 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:42.748393 (+ 3289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:42.749353 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:42.749362 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:42.749367 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:42.749533 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:42.750057 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:42.750064 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:42.750069 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:42.750488 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:42.750649 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:42.751307 (+ 658us) server_negotiation.cc:299] Negotiation successful
0925 09:26:42.751600 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":822,"thread_start_us":748,"threads_started":1}
I20260925 09:26:42.752719 12550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:42.729751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42763 (local address 127.10.114.65:37125)
0925 09:26:42.730573 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:42.730620 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:42.732731 (+ 2111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:42.733522 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:42.733548 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:42.734302 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:42.735123 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:42.735146 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:42.736870 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:42.736879 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:42.739051 (+ 2172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:42.739064 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:42.740211 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:42.740219 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:42.740429 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:42.741195 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:42.741231 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:42.744780 (+ 3549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:42.748554 (+ 3774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:42.748562 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:42.748581 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:42.749223 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:42.749694 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:42.749700 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:42.749703 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:42.749920 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:42.750637 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:42.750642 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:42.750931 (+ 289us) client_negotiation.cc:769] Sending connection context
0925 09:26:42.751158 (+ 227us) client_negotiation.cc:240] Negotiation successful
0925 09:26:42.751480 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":380,"threads_started":1}
I20260925 09:26:42.773532 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:42.758655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:49785 (local address 127.10.114.126:42763)
0925 09:26:42.758858 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:42.758867 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:42.759469 (+ 602us) server_negotiation.cc:407] Connection header received
0925 09:26:42.759628 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:42.759631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:42.759700 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:42.759823 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:42.761105 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:42.761817 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:42.764459 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:42.765132 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:42.765284 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:42.767201 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:42.767209 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:42.767219 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:42.767246 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:42.770333 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:42.771327 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:42.771330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:42.771332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:42.771469 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:42.772003 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:42.772006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:42.772008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:42.772254 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:42.772450 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:42.772947 (+ 497us) server_negotiation.cc:299] Negotiation successful
0925 09:26:42.773166 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 09:26:42.773592 12612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:42.758638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42763 (local address 127.10.114.65:49785)
0925 09:26:42.759338 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:42.759363 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:42.759521 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:42.759869 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:42.759875 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:42.760518 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:42.760917 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:42.760931 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:42.761990 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:42.762006 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:42.764232 (+ 2226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:42.764247 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:42.765342 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:42.765353 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:42.765576 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:42.766238 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:42.766255 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:42.767010 (+ 755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:42.770512 (+ 3502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:42.770520 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:42.770530 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:42.771136 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:42.771623 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:42.771630 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:42.771635 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:42.771856 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:42.772405 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:42.772412 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:42.772635 (+ 223us) client_negotiation.cc:769] Sending connection context
0925 09:26:42.772824 (+ 189us) client_negotiation.cc:240] Negotiation successful
0925 09:26:42.773184 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":358,"threads_started":1}
I20260925 09:26:42.780931 12525 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:35275
I20260925 09:26:42.781023 12660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:35275 every 8 connection(s)
I20260925 09:26:42.782089 12525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:26:42.784085 12661 heartbeater.cc:347] Connected to a master server at 127.10.114.126:42763
I20260925 09:26:42.784305 12661 heartbeater.cc:464] Registering TS with master...
I20260925 09:26:42.784881 12661 heartbeater.cc:511] Master 127.10.114.126:42763 requested a full tablet report, sending...
I20260925 09:26:42.786617 12454 ts_manager.cc:194] Registered new tserver with Master: 146f2508850b4ffd81f52247de045025 (127.10.114.65:35275)
I20260925 09:26:42.788125 12454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:37125
I20260925 09:26:42.790534 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 12525
I20260925 09:26:42.790758 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:26:42.791337 10697 external_mini_cluster.cc:1428] Setting key e7216fe1d60349c68e820b04f1f22a2a
WARNING: no policy specified for kudu/127.10.114.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
I20260925 09:26:42.831910 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42763
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:40449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:43.140563 12665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:43.140823 12665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:43.140986 12665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:43.147360 12665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:43.147480 12665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:43.147607 12665 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 09:26:43.147686 12665 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 09:26:43.147820 12665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:26:43.156507 12665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:40449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.114.126:42763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:43.157663 12665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:43.158905 12665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:43.170861 12670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:43.170782 12671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:43.170890 12673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:43.171542 12665 server_base.cc:1034] running on GCE node
I20260925 09:26:43.172204 12665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:43.173171 12665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:43.174381 12665 hybrid_clock.cc:648] HybridClock initialized: now 1790328403174329 us; error 45 us; skew 500 ppm
I20260925 09:26:43.174674 12665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:43 dist-test-slave-9pj8 krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328403, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:43.177613 12665 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:26:43.178912 12665 webserver.cc:460] Webserver started at http://127.10.114.66:37271/ using document root <none> and password file <none>
I20260925 09:26:43.179623 12665 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:43.179821 12665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:43.180143 12665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:43.183004 12665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "04faf4158bb24bfb8333efe88b849851"
format_stamp: "Formatted at 2026-09-25 09:26:43 on dist-test-slave-9pj8"
server_key: "dcaf890c600d23d6baac734779b46ba5"
server_key_iv: "6943bb4d50b6945cd7ba1465e3f7c96b"
server_key_version: "encryptionkey@0"
I20260925 09:26:43.183709 12665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "04faf4158bb24bfb8333efe88b849851"
format_stamp: "Formatted at 2026-09-25 09:26:43 on dist-test-slave-9pj8"
server_key: "dcaf890c600d23d6baac734779b46ba5"
server_key_iv: "6943bb4d50b6945cd7ba1465e3f7c96b"
server_key_version: "encryptionkey@0"
I20260925 09:26:43.188308 12665 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260925 09:26:43.191435 12680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:43.192660 12665 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 09:26:43.192800 12665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "04faf4158bb24bfb8333efe88b849851"
format_stamp: "Formatted at 2026-09-25 09:26:43 on dist-test-slave-9pj8"
server_key: "dcaf890c600d23d6baac734779b46ba5"
server_key_iv: "6943bb4d50b6945cd7ba1465e3f7c96b"
server_key_version: "encryptionkey@0"
I20260925 09:26:43.192978 12665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:43.207149 12665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:43.211457 12665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:43.211750 12665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:43.214469 12665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:26:43.214623 12665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:43.214746 12665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:26:43.214841 12665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 09:26:43 dist-test-slave-9pj8 krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328403, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:43.240305 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:43.215166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:52781 (local address 127.10.114.126:42763)
0925 09:26:43.215335 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:43.215340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:43.217021 (+ 1681us) server_negotiation.cc:407] Connection header received
0925 09:26:43.219411 (+ 2390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:43.219418 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:43.219545 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:43.219649 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:43.222638 (+ 2989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.223641 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.227136 (+ 3495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.227635 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.227798 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:43.233184 (+ 5386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:43.233188 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:43.233191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:43.233219 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:43.236315 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:43.237282 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:43.237289 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:43.237292 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:43.237401 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:43.237973 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:43.237978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:43.237981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:43.238295 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:43.238496 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:43.239677 (+ 1181us) server_negotiation.cc:299] Negotiation successful
0925 09:26:43.239944 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 09:26:43.242858 12689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:43.215867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42763 (local address 127.10.114.66:52781)
0925 09:26:43.216834 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:43.216893 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:43.219116 (+ 2223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:43.220159 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:43.220169 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:43.221221 (+ 1052us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:43.222398 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:43.222437 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.223848 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.223856 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:43.226894 (+ 3038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:43.226911 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.227967 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.227979 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:43.228386 (+ 407us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:43.229286 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:43.229327 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:43.232950 (+ 3623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:43.236486 (+ 3536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:43.236496 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:43.236524 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:43.237085 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:43.237511 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:43.237516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:43.237519 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:43.237864 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:43.238502 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:43.238518 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:43.239123 (+ 605us) client_negotiation.cc:769] Sending connection context
0925 09:26:43.239553 (+ 430us) client_negotiation.cc:240] Negotiation successful
0925 09:26:43.240290 (+ 737us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":398,"threads_started":1}
I20260925 09:26:43.257251 12665 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:39731
I20260925 09:26:43.257314 12799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:39731 every 8 connection(s)
I20260925 09:26:43.258592 12665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:26:43.260942 12800 heartbeater.cc:347] Connected to a master server at 127.10.114.126:42763
I20260925 09:26:43.261178 12800 heartbeater.cc:464] Registering TS with master...
I20260925 09:26:43.261905 12800 heartbeater.cc:511] Master 127.10.114.126:42763 requested a full tablet report, sending...
I20260925 09:26:43.263402 12454 ts_manager.cc:194] Registered new tserver with Master: 04faf4158bb24bfb8333efe88b849851 (127.10.114.66:39731)
I20260925 09:26:43.264279 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:43.250257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:47183 (local address 127.10.114.126:42763)
0925 09:26:43.250445 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:43.250453 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:43.250770 (+ 317us) server_negotiation.cc:407] Connection header received
0925 09:26:43.250974 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:43.250979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:43.251042 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:43.251136 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:43.252498 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.253462 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.255495 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.255979 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.256101 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:43.257927 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:43.257932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:43.257936 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:43.257980 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:43.261533 (+ 3553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:43.262315 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:43.262319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:43.262322 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:43.262454 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:43.262829 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:43.262838 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:43.262841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:43.263253 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:43.263376 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:43.263847 (+ 471us) server_negotiation.cc:299] Negotiation successful
0925 09:26:43.264013 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 09:26:43.264424 12783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:43.249712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42763 (local address 127.10.114.66:47183)
0925 09:26:43.250658 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:43.250680 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:43.250837 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:43.251169 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:43.251173 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:43.251754 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:43.252357 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:43.252376 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.253623 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.253630 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:43.255358 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:43.255369 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.256181 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.256187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:43.256374 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:43.256951 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:43.256967 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:43.257765 (+ 798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:43.261679 (+ 3914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:43.261683 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:43.261688 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:43.262209 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:43.262563 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:43.262567 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:43.262571 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:43.262719 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:43.263380 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:43.263388 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:43.263556 (+ 168us) client_negotiation.cc:769] Sending connection context
0925 09:26:43.263698 (+ 142us) client_negotiation.cc:240] Negotiation successful
0925 09:26:43.264014 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":448,"threads_started":1}
I20260925 09:26:43.264535 12454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:52781
I20260925 09:26:43.268007 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 12665
I20260925 09:26:43.268186 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:26:43.268592 10697 external_mini_cluster.cc:1428] Setting key f685a3264a2709fc9086596d539e418f
WARNING: no policy specified for kudu/127.10.114.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
I20260925 09:26:43.306614 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42763
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:40449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:43.622437 12804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:43.622704 12804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:43.622823 12804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:43.629352 12804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:43.629437 12804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:43.629515 12804 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 09:26:43.629565 12804 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 09:26:43.629635 12804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:26:43.638891 12804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:40449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.114.126:42763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:43.640100 12804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:43.641356 12804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:43.654038 12812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:43.654016 12810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:43.654327 12809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:43.654594 12804 server_base.cc:1034] running on GCE node
I20260925 09:26:43.655375 12804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:43.656265 12804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:43.657455 12804 hybrid_clock.cc:648] HybridClock initialized: now 1790328403657407 us; error 33 us; skew 500 ppm
I20260925 09:26:43.657724 12804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:43 dist-test-slave-9pj8 krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328403, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:43.660579 12804 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:26:43.661861 12804 webserver.cc:460] Webserver started at http://127.10.114.67:34415/ using document root <none> and password file <none>
I20260925 09:26:43.662472 12804 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:43.662590 12804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:43.662900 12804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:43.665656 12804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "739baee21b8141b6a16b9b553c7c26b3"
format_stamp: "Formatted at 2026-09-25 09:26:43 on dist-test-slave-9pj8"
server_key: "bcf6a5e532649cccc19858c8baf551e9"
server_key_iv: "d3856f2f481c53aa98a9b235c2e1970a"
server_key_version: "encryptionkey@0"
I20260925 09:26:43.666460 12804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "739baee21b8141b6a16b9b553c7c26b3"
format_stamp: "Formatted at 2026-09-25 09:26:43 on dist-test-slave-9pj8"
server_key: "bcf6a5e532649cccc19858c8baf551e9"
server_key_iv: "d3856f2f481c53aa98a9b235c2e1970a"
server_key_version: "encryptionkey@0"
I20260925 09:26:43.671307 12804 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:26:43.675436 12819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:43.676775 12804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:26:43.676965 12804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "739baee21b8141b6a16b9b553c7c26b3"
format_stamp: "Formatted at 2026-09-25 09:26:43 on dist-test-slave-9pj8"
server_key: "bcf6a5e532649cccc19858c8baf551e9"
server_key_iv: "d3856f2f481c53aa98a9b235c2e1970a"
server_key_version: "encryptionkey@0"
I20260925 09:26:43.677136 12804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:43.698367 12804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:43.702229 12804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:43.702485 12804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:43.704857 12804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:26:43.704984 12804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:43.705096 12804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:26:43.705186 12804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 09:26:43 dist-test-slave-9pj8 krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328403, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:43.729192 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:43.705544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:51729 (local address 127.10.114.126:42763)
0925 09:26:43.705730 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:43.705736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:43.707217 (+ 1481us) server_negotiation.cc:407] Connection header received
0925 09:26:43.709882 (+ 2665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:43.709890 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:43.709992 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:43.710130 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:43.712621 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.713715 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.716008 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.716691 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.716831 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:43.722070 (+ 5239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:43.722077 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:43.722083 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:43.722137 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:43.725681 (+ 3544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:43.726608 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:43.726613 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:43.726616 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:43.726790 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:43.727443 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:43.727447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:43.727452 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:43.727811 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:43.727979 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:43.728634 (+ 655us) server_negotiation.cc:299] Negotiation successful
0925 09:26:43.728853 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 09:26:43.730057 12828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:43.706114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42763 (local address 127.10.114.67:51729)
0925 09:26:43.707019 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:43.707067 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:43.709470 (+ 2403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:43.710459 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:43.710466 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:43.711274 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:43.712445 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:43.712467 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.713915 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.713920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:43.715841 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:43.715855 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.716945 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.716969 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:43.717180 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:43.717981 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:43.718010 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:43.721728 (+ 3718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:43.725847 (+ 4119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:43.725852 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:43.725866 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:43.726470 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:43.726928 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:43.726936 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:43.726946 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:43.727281 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:43.727963 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:43.727970 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:43.728281 (+ 311us) client_negotiation.cc:769] Sending connection context
0925 09:26:43.728520 (+ 239us) client_negotiation.cc:240] Negotiation successful
0925 09:26:43.728994 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":407,"threads_started":1}
I20260925 09:26:43.750059 12899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:43.736670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42763 (local address 127.10.114.67:45709)
0925 09:26:43.737331 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:43.737345 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:43.737481 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:43.737765 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:43.737769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:43.738239 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:43.738520 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:43.738527 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.739418 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.739422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:43.740631 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:43.740641 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.741550 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.741557 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:43.741771 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:43.742507 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:43.742534 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:43.743559 (+ 1025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:43.747012 (+ 3453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:43.747035 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:43.747038 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:43.747548 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:43.748018 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:43.748026 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:43.748029 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:43.748151 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:43.748731 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:43.748736 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:43.748943 (+ 207us) client_negotiation.cc:769] Sending connection context
0925 09:26:43.749191 (+ 248us) client_negotiation.cc:240] Negotiation successful
0925 09:26:43.749512 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":635,"threads_started":1}
I20260925 09:26:43.750746 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:43.737068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:45709 (local address 127.10.114.126:42763)
0925 09:26:43.737302 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:43.737309 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:43.737423 (+ 114us) server_negotiation.cc:407] Connection header received
0925 09:26:43.737580 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:43.737585 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:43.737658 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:43.737747 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:43.738642 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.739272 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.740754 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.741366 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.741483 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:43.743739 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:43.743745 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:43.743750 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:43.743811 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:43.746891 (+ 3080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:43.747668 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:43.747675 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:43.747680 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:43.747886 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:43.748272 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:43.748277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:43.748283 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:43.748639 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:43.748763 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:43.750314 (+ 1551us) server_negotiation.cc:299] Negotiation successful
0925 09:26:43.750486 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 09:26:43.755584 12804 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:41333
I20260925 09:26:43.755661 12938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:41333 every 8 connection(s)
I20260925 09:26:43.756906 12804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:26:43.758797 12939 heartbeater.cc:347] Connected to a master server at 127.10.114.126:42763
I20260925 09:26:43.759001 12939 heartbeater.cc:464] Registering TS with master...
I20260925 09:26:43.759585 12939 heartbeater.cc:511] Master 127.10.114.126:42763 requested a full tablet report, sending...
I20260925 09:26:43.760969 12453 ts_manager.cc:194] Registered new tserver with Master: 739baee21b8141b6a16b9b553c7c26b3 (127.10.114.67:41333)
I20260925 09:26:43.761792 12453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:51729
I20260925 09:26:43.762554 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 12804
I20260925 09:26:43.762758 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:26:43.763159 10697 external_mini_cluster.cc:1428] Setting key 96dc8fcf184eb6e6ebb272e290df7bc3
I20260925 09:26:43.765859 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 09:26:43 dist-test-slave-9pj8 krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260925 09:26:43.790400 12661 heartbeater.cc:503] Master 127.10.114.126:42763 was elected leader, sending a full tablet report...
Sep 25 09:26:43 dist-test-slave-9pj8 krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:43.810185 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:43.797231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50710 (local address 127.10.114.126:42763)
0925 09:26:43.797456 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:43.797465 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:43.797742 (+ 277us) server_negotiation.cc:407] Connection header received
0925 09:26:43.797873 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:43.797876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:43.797931 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:43.798008 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:43.799221 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.799981 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.801169 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.801519 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.801600 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:43.805478 (+ 3878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:43.805482 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:43.805497 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:43.805524 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:43.808008 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:43.808616 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:43.808620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:43.808622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:43.808748 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:43.809060 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:43.809063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:43.809065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:43.809289 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:43.809440 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:43.809717 (+ 277us) server_negotiation.cc:299] Negotiation successful
0925 09:26:43.809847 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260925 09:26:43.813862 12453 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:26:43.820072 12453 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:26:43.851540 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:43.832226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45308 (local address 127.10.114.66:39731)
0925 09:26:43.843200 (+ 10974us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:43.843210 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:43.844049 (+ 839us) server_negotiation.cc:407] Connection header received
0925 09:26:43.844183 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:43.844194 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:43.844492 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:43.844780 (+ 288us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:43.845507 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.846828 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.849005 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.850300 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.850416 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:43.850598 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:43.851103 (+ 505us) server_negotiation.cc:299] Negotiation successful
0925 09:26:43.851279 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10773,"thread_start_us":10983,"threads_started":1}
I20260925 09:26:43.851589 12956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:43.831819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:39731 (local address 127.0.0.1:45308)
0925 09:26:43.843404 (+ 11585us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:43.843432 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:43.843570 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:43.844752 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:43.844759 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:43.844787 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:43.845234 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:43.845260 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.846972 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.846977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:43.848843 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:43.848874 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.850702 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.850708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:43.850879 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:43.850925 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 09:26:43.851051 (+ 126us) client_negotiation.cc:240] Negotiation successful
0925 09:26:43.851218 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11488,"mutex_wait_us":3,"thread_start_us":562,"threads_started":1}
I20260925 09:26:43.852672 12954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:43.831815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57892 (local address 127.10.114.65:35275)
0925 09:26:43.845404 (+ 13589us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:43.845409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:43.845454 (+ 45us) server_negotiation.cc:407] Connection header received
0925 09:26:43.845543 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:43.845550 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:43.845742 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:43.845949 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:43.846530 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.847891 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.850082 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.851368 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.851487 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:43.851683 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:43.852253 (+ 570us) server_negotiation.cc:299] Negotiation successful
0925 09:26:43.852391 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13457,"thread_start_us":13323,"threads_started":1}
I20260925 09:26:43.852715 12952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:43.835090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:41333 (local address 127.0.0.1:34976)
0925 09:26:43.843763 (+ 8673us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:43.843812 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:43.843912 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:43.845050 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:43.845054 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:43.845073 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:43.845602 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:43.845611 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.847484 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.847491 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:43.849961 (+ 2470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:43.849974 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.851986 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.851992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:43.852208 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:43.852251 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 09:26:43.852319 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 09:26:43.852425 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8518,"mutex_wait_us":1086,"thread_start_us":8047,"threads_started":1}
I20260925 09:26:43.853260 12957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:43.835730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34976 (local address 127.10.114.67:41333)
0925 09:26:43.844030 (+ 8300us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:43.844036 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:43.844085 (+ 49us) server_negotiation.cc:407] Connection header received
0925 09:26:43.844177 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:43.844184 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:43.844450 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:43.844780 (+ 330us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:43.845846 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.847347 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.850143 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.851318 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.851424 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:43.851634 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:43.852800 (+ 1166us) server_negotiation.cc:299] Negotiation successful
0925 09:26:43.852939 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8168,"thread_start_us":7955,"threads_started":1}
I20260925 09:26:43.853565 12953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:43.831566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:35275 (local address 127.0.0.1:57892)
0925 09:26:43.832672 (+ 1106us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:43.832774 (+ 102us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:43.833010 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:43.845933 (+ 12923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:43.845940 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:43.845968 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:43.846372 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:43.846383 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.848023 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.848030 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:43.849913 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:43.849926 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:43.851824 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:43.851832 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:43.852044 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:43.852075 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 09:26:43.852178 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 09:26:43.852314 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"mutex_wait_us":3,"thread_start_us":12089,"threads_started":1}
I20260925 09:26:43.855250 12730 tablet_service.cc:1467] Processing CreateTablet for tablet 07b5c7fd2b0c4446b9d22281d991cc44 (DEFAULT_TABLE table=test-table [id=826cf9b5a1024199838f15fa02f43053]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:43.856431 12730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07b5c7fd2b0c4446b9d22281d991cc44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:43.856515 12591 tablet_service.cc:1467] Processing CreateTablet for tablet 07b5c7fd2b0c4446b9d22281d991cc44 (DEFAULT_TABLE table=test-table [id=826cf9b5a1024199838f15fa02f43053]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:43.856474 12869 tablet_service.cc:1467] Processing CreateTablet for tablet 07b5c7fd2b0c4446b9d22281d991cc44 (DEFAULT_TABLE table=test-table [id=826cf9b5a1024199838f15fa02f43053]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:43.857718 12591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07b5c7fd2b0c4446b9d22281d991cc44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:43.857726 12869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07b5c7fd2b0c4446b9d22281d991cc44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:43.866961 12958 tablet_bootstrap.cc:492] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851: Bootstrap starting.
I20260925 09:26:43.869626 12958 tablet_bootstrap.cc:654] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:43.870647 12959 tablet_bootstrap.cc:492] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3: Bootstrap starting.
I20260925 09:26:43.870731 12958 log.cc:824] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851: Log is configured to *not* fsync() on all Append() calls
I20260925 09:26:43.872265 12960 tablet_bootstrap.cc:492] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025: Bootstrap starting.
I20260925 09:26:43.874513 12959 tablet_bootstrap.cc:654] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:43.875476 12960 tablet_bootstrap.cc:654] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:43.875715 12959 log.cc:824] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3: Log is configured to *not* fsync() on all Append() calls
I20260925 09:26:43.875939 12958 tablet_bootstrap.cc:492] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851: No bootstrap required, opened a new log
I20260925 09:26:43.876196 12958 ts_tablet_manager.cc:1397] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260925 09:26:43.876657 12960 log.cc:824] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025: Log is configured to *not* fsync() on all Append() calls
I20260925 09:26:43.878513 12959 tablet_bootstrap.cc:492] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3: No bootstrap required, opened a new log
I20260925 09:26:43.878759 12959 ts_tablet_manager.cc:1397] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260925 09:26:43.879024 12960 tablet_bootstrap.cc:492] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025: No bootstrap required, opened a new log
I20260925 09:26:43.879256 12960 ts_tablet_manager.cc:1397] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.002s
I20260925 09:26:43.886536 12960 raft_consensus.cc:348] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:43.886894 12960 raft_consensus.cc:374] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:43.886767 12958 raft_consensus.cc:348] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:43.887033 12960 raft_consensus.cc:729] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 146f2508850b4ffd81f52247de045025, State: Initialized, Role: FOLLOWER
I20260925 09:26:43.887188 12958 raft_consensus.cc:374] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:43.887282 12958 raft_consensus.cc:729] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04faf4158bb24bfb8333efe88b849851, State: Initialized, Role: FOLLOWER
I20260925 09:26:43.887681 12960 consensus_queue.cc:260] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:43.888029 12958 consensus_queue.cc:260] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:43.889285 12960 ts_tablet_manager.cc:1428] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260925 09:26:43.889884 12959 raft_consensus.cc:348] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:43.890218 12959 raft_consensus.cc:374] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:43.890309 12959 raft_consensus.cc:729] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 739baee21b8141b6a16b9b553c7c26b3, State: Initialized, Role: FOLLOWER
I20260925 09:26:43.890834 12959 consensus_queue.cc:260] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:43.894407 12959 ts_tablet_manager.cc:1428] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260925 09:26:43.894836 12939 heartbeater.cc:503] Master 127.10.114.126:42763 was elected leader, sending a full tablet report...
I20260925 09:26:43.895215 12800 heartbeater.cc:503] Master 127.10.114.126:42763 was elected leader, sending a full tablet report...
I20260925 09:26:43.896274 12958 ts_tablet_manager.cc:1428] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851: Time spent starting tablet: real 0.020s user 0.013s sys 0.005s
W20260925 09:26:44.008602 12940 tablet.cc:2369] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:26:44.010708 12801 tablet.cc:2369] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:26:44.019554 12966 raft_consensus.cc:482] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:26:44.019940 12966 raft_consensus.cc:504] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:44.021850 12966 leader_election.cc:288] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 146f2508850b4ffd81f52247de045025 (127.10.114.65:35275), 04faf4158bb24bfb8333efe88b849851 (127.10.114.66:39731)
I20260925 09:26:44.028649 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:44.022740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:45003 (local address 127.10.114.66:39731)
0925 09:26:44.022988 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:44.022997 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:44.023019 (+ 22us) server_negotiation.cc:407] Connection header received
0925 09:26:44.023101 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:44.023107 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:44.023194 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:44.023318 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:44.023810 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.024738 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.026792 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.027658 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.027825 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:44.027886 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:44.028303 (+ 417us) server_negotiation.cc:299] Negotiation successful
0925 09:26:44.028390 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260925 09:26:44.028882 12828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:44.022224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:39731 (local address 127.10.114.67:45003)
0925 09:26:44.022467 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:44.022493 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:44.022659 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:44.023366 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:44.023372 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:44.023406 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:44.023616 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:44.023639 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.024910 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.024917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:44.026642 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:44.026655 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.027847 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.027856 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:44.028120 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:44.028165 (+ 45us) client_negotiation.cc:769] Sending connection context
0925 09:26:44.028285 (+ 120us) client_negotiation.cc:240] Negotiation successful
0925 09:26:44.028425 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":32}
I20260925 09:26:44.029347 12954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:44.022457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:53027 (local address 127.10.114.65:35275)
0925 09:26:44.022679 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:44.022686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:44.023334 (+ 648us) server_negotiation.cc:407] Connection header received
0925 09:26:44.023498 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:44.023503 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:44.023579 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:44.023723 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:44.024392 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.025435 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.027250 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.028397 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.028561 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:44.028661 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:44.028994 (+ 333us) server_negotiation.cc:299] Negotiation successful
0925 09:26:44.029124 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260925 09:26:44.030049 12967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:44.022344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:35275 (local address 127.10.114.67:53027)
0925 09:26:44.023206 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:44.023226 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:44.023360 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:44.023814 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:44.023818 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:44.023842 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:44.024194 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:44.024205 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.025579 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.025582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:44.027084 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:44.027096 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.028611 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.028617 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:44.028812 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:44.028854 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 09:26:44.029568 (+ 714us) client_negotiation.cc:240] Negotiation successful
0925 09:26:44.029696 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"mutex_wait_us":41,"thread_start_us":524,"threads_started":1}
I20260925 09:26:44.029925 12750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07b5c7fd2b0c4446b9d22281d991cc44" candidate_uuid: "739baee21b8141b6a16b9b553c7c26b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04faf4158bb24bfb8333efe88b849851" is_pre_election: true
I20260925 09:26:44.030460 12750 raft_consensus.cc:2454] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 739baee21b8141b6a16b9b553c7c26b3 in term 0.
I20260925 09:26:44.031167 12820 leader_election.cc:302] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04faf4158bb24bfb8333efe88b849851, 739baee21b8141b6a16b9b553c7c26b3; no voters:
I20260925 09:26:44.031239 12616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07b5c7fd2b0c4446b9d22281d991cc44" candidate_uuid: "739baee21b8141b6a16b9b553c7c26b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146f2508850b4ffd81f52247de045025" is_pre_election: true
I20260925 09:26:44.031621 12966 raft_consensus.cc:2781] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:26:44.031838 12966 raft_consensus.cc:482] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:26:44.031965 12966 raft_consensus.cc:3037] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:44.031858 12616 raft_consensus.cc:2454] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 739baee21b8141b6a16b9b553c7c26b3 in term 0.
I20260925 09:26:44.034338 12966 raft_consensus.cc:504] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
W20260925 09:26:44.034569 12662 tablet.cc:2369] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:26:44.034940 12966 leader_election.cc:288] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [CANDIDATE]: Term 1 election: Requested vote from peers 146f2508850b4ffd81f52247de045025 (127.10.114.65:35275), 04faf4158bb24bfb8333efe88b849851 (127.10.114.66:39731)
I20260925 09:26:44.035415 12750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07b5c7fd2b0c4446b9d22281d991cc44" candidate_uuid: "739baee21b8141b6a16b9b553c7c26b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04faf4158bb24bfb8333efe88b849851"
I20260925 09:26:44.035409 12616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07b5c7fd2b0c4446b9d22281d991cc44" candidate_uuid: "739baee21b8141b6a16b9b553c7c26b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146f2508850b4ffd81f52247de045025"
I20260925 09:26:44.035590 12750 raft_consensus.cc:3037] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:44.035590 12616 raft_consensus.cc:3037] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:44.038601 12750 raft_consensus.cc:2454] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 739baee21b8141b6a16b9b553c7c26b3 in term 1.
I20260925 09:26:44.038601 12616 raft_consensus.cc:2454] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 739baee21b8141b6a16b9b553c7c26b3 in term 1.
I20260925 09:26:44.038935 12820 leader_election.cc:302] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04faf4158bb24bfb8333efe88b849851, 739baee21b8141b6a16b9b553c7c26b3; no voters:
I20260925 09:26:44.039204 12966 raft_consensus.cc:2781] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:26:44.039568 12966 raft_consensus.cc:686] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [term 1 LEADER]: Becoming Leader. State: Replica: 739baee21b8141b6a16b9b553c7c26b3, State: Running, Role: LEADER
I20260925 09:26:44.040043 12966 consensus_queue.cc:237] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:44.044459 12453 catalog_manager.cc:5495] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 739baee21b8141b6a16b9b553c7c26b3 (127.10.114.67). New cstate: current_term: 1 leader_uuid: "739baee21b8141b6a16b9b553c7c26b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } health_report { overall_health: HEALTHY } } }
Sep 25 09:26:44 dist-test-slave-9pj8 krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328402, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:44.114250 12975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:44.101939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42763 (local address 127.0.0.1:50722)
0925 09:26:44.102444 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:44.102458 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:44.102567 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:44.102864 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:44.102867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:44.103579 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:44.103908 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:44.103933 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.104942 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.104946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:44.105943 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:44.105954 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.106666 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.106670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:44.106788 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:44.107460 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:44.107476 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:44.109598 (+ 2122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:44.112205 (+ 2607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:44.112214 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:44.112219 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:44.112565 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:44.112869 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:44.112872 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:44.112874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:44.112994 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:44.113423 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:44.113428 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:44.113566 (+ 138us) client_negotiation.cc:769] Sending connection context
0925 09:26:44.113710 (+ 144us) client_negotiation.cc:240] Negotiation successful
0925 09:26:44.113892 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":271,"threads_started":1}
I20260925 09:26:44.114253 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:44.101825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50722 (local address 127.10.114.126:42763)
0925 09:26:44.101968 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:44.101972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:44.102577 (+ 605us) server_negotiation.cc:407] Connection header received
0925 09:26:44.102673 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:44.102676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:44.102747 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:44.102838 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:44.104102 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.104809 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.106072 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.106478 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.106566 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:44.109740 (+ 3174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:44.109744 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:44.109747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:44.109773 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:44.112090 (+ 2317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:44.112659 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:44.112662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:44.112664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:44.112779 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:44.113075 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:44.113078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:44.113080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:44.113321 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:44.113423 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:44.113797 (+ 374us) server_negotiation.cc:299] Negotiation successful
0925 09:26:44.114003 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":45}
I20260925 09:26:44.118949 12453 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:50722:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 09:26:44.120033 12453 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:26:44.125976 12730 tablet_service.cc:1467] Processing CreateTablet for tablet fb48215b0f544acbb4bf43ae905728fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b72cc617d1d6435c89bd210f398d98c9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 09:26:44.125893 12591 tablet_service.cc:1467] Processing CreateTablet for tablet fb48215b0f544acbb4bf43ae905728fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b72cc617d1d6435c89bd210f398d98c9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 09:26:44.126266 12869 tablet_service.cc:1467] Processing CreateTablet for tablet fb48215b0f544acbb4bf43ae905728fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b72cc617d1d6435c89bd210f398d98c9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 09:26:44.126775 12730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb48215b0f544acbb4bf43ae905728fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:44.126799 12591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb48215b0f544acbb4bf43ae905728fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:44.126950 12869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb48215b0f544acbb4bf43ae905728fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:44.133527 12959 tablet_bootstrap.cc:492] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3: Bootstrap starting.
I20260925 09:26:44.133569 12958 tablet_bootstrap.cc:492] T fb48215b0f544acbb4bf43ae905728fc P 04faf4158bb24bfb8333efe88b849851: Bootstrap starting.
I20260925 09:26:44.135102 12959 tablet_bootstrap.cc:654] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:44.135102 12958 tablet_bootstrap.cc:654] T fb48215b0f544acbb4bf43ae905728fc P 04faf4158bb24bfb8333efe88b849851: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:44.136965 12960 tablet_bootstrap.cc:492] T fb48215b0f544acbb4bf43ae905728fc P 146f2508850b4ffd81f52247de045025: Bootstrap starting.
I20260925 09:26:44.137375 12959 tablet_bootstrap.cc:492] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3: No bootstrap required, opened a new log
I20260925 09:26:44.137501 12959 ts_tablet_manager.cc:1397] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260925 09:26:44.138512 12960 tablet_bootstrap.cc:654] T fb48215b0f544acbb4bf43ae905728fc P 146f2508850b4ffd81f52247de045025: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:44.138486 12959 raft_consensus.cc:348] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:44.138739 12959 raft_consensus.cc:374] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:44.138846 12959 raft_consensus.cc:729] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 739baee21b8141b6a16b9b553c7c26b3, State: Initialized, Role: FOLLOWER
I20260925 09:26:44.139043 12958 tablet_bootstrap.cc:492] T fb48215b0f544acbb4bf43ae905728fc P 04faf4158bb24bfb8333efe88b849851: No bootstrap required, opened a new log
I20260925 09:26:44.139173 12958 ts_tablet_manager.cc:1397] T fb48215b0f544acbb4bf43ae905728fc P 04faf4158bb24bfb8333efe88b849851: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260925 09:26:44.139112 12959 consensus_queue.cc:260] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:44.139580 12966 tablet_replica.cc:439] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } } }
I20260925 09:26:44.139932 12966 tablet_replica.cc:442] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:26:44.140763 12958 raft_consensus.cc:348] T fb48215b0f544acbb4bf43ae905728fc P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:44.141024 12958 raft_consensus.cc:374] T fb48215b0f544acbb4bf43ae905728fc P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:44.141120 12958 raft_consensus.cc:729] T fb48215b0f544acbb4bf43ae905728fc P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04faf4158bb24bfb8333efe88b849851, State: Initialized, Role: FOLLOWER
I20260925 09:26:44.141371 12958 consensus_queue.cc:260] T fb48215b0f544acbb4bf43ae905728fc P 04faf4158bb24bfb8333efe88b849851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:44.141963 12958 ts_tablet_manager.cc:1428] T fb48215b0f544acbb4bf43ae905728fc P 04faf4158bb24bfb8333efe88b849851: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 09:26:44.141826 12965 tablet_replica.cc:439] T fb48215b0f544acbb4bf43ae905728fc P 04faf4158bb24bfb8333efe88b849851: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } } }
I20260925 09:26:44.142228 12965 tablet_replica.cc:442] T fb48215b0f544acbb4bf43ae905728fc P 04faf4158bb24bfb8333efe88b849851: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:26:44.142743 12959 ts_tablet_manager.cc:1428] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260925 09:26:44.144485 12960 tablet_bootstrap.cc:492] T fb48215b0f544acbb4bf43ae905728fc P 146f2508850b4ffd81f52247de045025: No bootstrap required, opened a new log
I20260925 09:26:44.144615 12960 ts_tablet_manager.cc:1397] T fb48215b0f544acbb4bf43ae905728fc P 146f2508850b4ffd81f52247de045025: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.004s
I20260925 09:26:44.145289 12960 raft_consensus.cc:348] T fb48215b0f544acbb4bf43ae905728fc P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:44.145542 12960 raft_consensus.cc:374] T fb48215b0f544acbb4bf43ae905728fc P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:44.145633 12960 raft_consensus.cc:729] T fb48215b0f544acbb4bf43ae905728fc P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 146f2508850b4ffd81f52247de045025, State: Initialized, Role: FOLLOWER
I20260925 09:26:44.145834 12960 consensus_queue.cc:260] T fb48215b0f544acbb4bf43ae905728fc P 146f2508850b4ffd81f52247de045025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:44.146440 12960 ts_tablet_manager.cc:1428] T fb48215b0f544acbb4bf43ae905728fc P 146f2508850b4ffd81f52247de045025: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260925 09:26:44.146317 12964 tablet_replica.cc:439] T fb48215b0f544acbb4bf43ae905728fc P 146f2508850b4ffd81f52247de045025: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } } }
I20260925 09:26:44.146700 12964 tablet_replica.cc:442] T fb48215b0f544acbb4bf43ae905728fc P 146f2508850b4ffd81f52247de045025: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:26:44.285904 12966 raft_consensus.cc:482] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:26:44.286164 12966 raft_consensus.cc:504] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:44.286811 12966 leader_election.cc:288] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 146f2508850b4ffd81f52247de045025 (127.10.114.65:35275), 04faf4158bb24bfb8333efe88b849851 (127.10.114.66:39731)
I20260925 09:26:44.287235 12616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb48215b0f544acbb4bf43ae905728fc" candidate_uuid: "739baee21b8141b6a16b9b553c7c26b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146f2508850b4ffd81f52247de045025" is_pre_election: true
I20260925 09:26:44.287338 12750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb48215b0f544acbb4bf43ae905728fc" candidate_uuid: "739baee21b8141b6a16b9b553c7c26b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04faf4158bb24bfb8333efe88b849851" is_pre_election: true
I20260925 09:26:44.287477 12616 raft_consensus.cc:2454] T fb48215b0f544acbb4bf43ae905728fc P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 739baee21b8141b6a16b9b553c7c26b3 in term 0.
I20260925 09:26:44.287642 12750 raft_consensus.cc:2454] T fb48215b0f544acbb4bf43ae905728fc P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 739baee21b8141b6a16b9b553c7c26b3 in term 0.
I20260925 09:26:44.287883 12822 leader_election.cc:302] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 146f2508850b4ffd81f52247de045025, 739baee21b8141b6a16b9b553c7c26b3; no voters:
I20260925 09:26:44.288149 12966 raft_consensus.cc:2781] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:26:44.288232 12966 raft_consensus.cc:482] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:26:44.288355 12966 raft_consensus.cc:3037] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:44.290218 12966 raft_consensus.cc:504] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:44.290817 12966 leader_election.cc:288] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3 [CANDIDATE]: Term 1 election: Requested vote from peers 146f2508850b4ffd81f52247de045025 (127.10.114.65:35275), 04faf4158bb24bfb8333efe88b849851 (127.10.114.66:39731)
I20260925 09:26:44.291211 12750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb48215b0f544acbb4bf43ae905728fc" candidate_uuid: "739baee21b8141b6a16b9b553c7c26b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04faf4158bb24bfb8333efe88b849851"
I20260925 09:26:44.291195 12616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb48215b0f544acbb4bf43ae905728fc" candidate_uuid: "739baee21b8141b6a16b9b553c7c26b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146f2508850b4ffd81f52247de045025"
I20260925 09:26:44.291437 12750 raft_consensus.cc:3037] T fb48215b0f544acbb4bf43ae905728fc P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:44.291437 12616 raft_consensus.cc:3037] T fb48215b0f544acbb4bf43ae905728fc P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:44.294557 12616 raft_consensus.cc:2454] T fb48215b0f544acbb4bf43ae905728fc P 146f2508850b4ffd81f52247de045025 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 739baee21b8141b6a16b9b553c7c26b3 in term 1.
I20260925 09:26:44.294557 12750 raft_consensus.cc:2454] T fb48215b0f544acbb4bf43ae905728fc P 04faf4158bb24bfb8333efe88b849851 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 739baee21b8141b6a16b9b553c7c26b3 in term 1.
I20260925 09:26:44.294914 12820 leader_election.cc:302] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04faf4158bb24bfb8333efe88b849851, 739baee21b8141b6a16b9b553c7c26b3; no voters:
I20260925 09:26:44.295185 12966 raft_consensus.cc:2781] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:26:44.295423 12966 raft_consensus.cc:686] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3 [term 1 LEADER]: Becoming Leader. State: Replica: 739baee21b8141b6a16b9b553c7c26b3, State: Running, Role: LEADER
I20260925 09:26:44.295696 12966 consensus_queue.cc:237] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:44.297585 12968 tablet_replica.cc:439] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3: TxnStatusTablet state changed. Reason: New leader 739baee21b8141b6a16b9b553c7c26b3. Latest consensus state: current_term: 1 leader_uuid: "739baee21b8141b6a16b9b553c7c26b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } } }
I20260925 09:26:44.297829 12968 tablet_replica.cc:442] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3: This TxnStatusTablet replica's current role is: LEADER
I20260925 09:26:44.298465 12980 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 09:26:44.298753 12980 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 09:26:44.299105 12453 catalog_manager.cc:5495] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 739baee21b8141b6a16b9b553c7c26b3 (127.10.114.67). New cstate: current_term: 1 leader_uuid: "739baee21b8141b6a16b9b553c7c26b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } health_report { overall_health: HEALTHY } } }
I20260925 09:26:44.335129 12957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:44.329389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34988 (local address 127.10.114.67:41333)
0925 09:26:44.329560 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:44.329567 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:44.329669 (+ 102us) server_negotiation.cc:407] Connection header received
0925 09:26:44.329947 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:44.329951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:44.330020 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:44.330171 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:26:44.330488 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.331207 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.332816 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.333366 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.333501 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:44.333982 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:26:44.334477 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:26:44.334632 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:44.334739 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 09:26:44.334815 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 09:26:44.335148 12975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:44.329318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:41333 (local address 127.0.0.1:34988)
0925 09:26:44.329564 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:44.329585 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:44.329754 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:44.330174 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:44.330178 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:44.330190 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:26:44.330364 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:44.330372 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.331327 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.331334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:44.332667 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:44.332684 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.333535 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.333542 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:44.333726 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:44.333776 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:26:44.334592 (+ 816us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:26:44.334612 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 09:26:44.334704 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 09:26:44.334849 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260925 09:26:44.342634 12616 raft_consensus.cc:1261] T fb48215b0f544acbb4bf43ae905728fc P 146f2508850b4ffd81f52247de045025 [term 1 FOLLOWER]: Refusing update from remote peer 739baee21b8141b6a16b9b553c7c26b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:26:44.342634 12750 raft_consensus.cc:1261] T fb48215b0f544acbb4bf43ae905728fc P 04faf4158bb24bfb8333efe88b849851 [term 1 FOLLOWER]: Refusing update from remote peer 739baee21b8141b6a16b9b553c7c26b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:26:44.343591 12966 consensus_queue.cc:1035] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:26:44.343907 12968 consensus_queue.cc:1035] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:26:44.354116 12964 tablet_replica.cc:439] T fb48215b0f544acbb4bf43ae905728fc P 146f2508850b4ffd81f52247de045025: TxnStatusTablet state changed. Reason: New leader 739baee21b8141b6a16b9b553c7c26b3. Latest consensus state: current_term: 1 leader_uuid: "739baee21b8141b6a16b9b553c7c26b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } } }
I20260925 09:26:44.354417 12964 tablet_replica.cc:442] T fb48215b0f544acbb4bf43ae905728fc P 146f2508850b4ffd81f52247de045025: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:26:44.357970 12965 tablet_replica.cc:439] T fb48215b0f544acbb4bf43ae905728fc P 04faf4158bb24bfb8333efe88b849851: TxnStatusTablet state changed. Reason: New leader 739baee21b8141b6a16b9b553c7c26b3. Latest consensus state: current_term: 1 leader_uuid: "739baee21b8141b6a16b9b553c7c26b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } } }
I20260925 09:26:44.358251 12965 tablet_replica.cc:442] T fb48215b0f544acbb4bf43ae905728fc P 04faf4158bb24bfb8333efe88b849851: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:26:44.358985 12983 mvcc.cc:205] Tried to move back new op lower bound from 7333185144177516544 to 7333185143998148608. Current Snapshot: MvccSnapshot[applied={T|T < 7333185144177516544}]
I20260925 09:26:44.359799 12966 tablet_replica.cc:439] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "739baee21b8141b6a16b9b553c7c26b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } } }
I20260925 09:26:44.360073 12966 tablet_replica.cc:442] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3: This TxnStatusTablet replica's current role is: LEADER
I20260925 09:26:44.360302 12966 tablet_replica.cc:439] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "739baee21b8141b6a16b9b553c7c26b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } } }
I20260925 09:26:44.360546 12966 tablet_replica.cc:442] T fb48215b0f544acbb4bf43ae905728fc P 739baee21b8141b6a16b9b553c7c26b3: This TxnStatusTablet replica's current role is: LEADER
I20260925 09:26:44.372578 12965 tablet_replica.cc:439] T fb48215b0f544acbb4bf43ae905728fc P 04faf4158bb24bfb8333efe88b849851: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "739baee21b8141b6a16b9b553c7c26b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } } }
I20260925 09:26:44.372920 12965 tablet_replica.cc:442] T fb48215b0f544acbb4bf43ae905728fc P 04faf4158bb24bfb8333efe88b849851: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:26:44.373459 12964 tablet_replica.cc:439] T fb48215b0f544acbb4bf43ae905728fc P 146f2508850b4ffd81f52247de045025: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "739baee21b8141b6a16b9b553c7c26b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } } }
I20260925 09:26:44.373667 12964 tablet_replica.cc:442] T fb48215b0f544acbb4bf43ae905728fc P 146f2508850b4ffd81f52247de045025: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:26:44.384162 12957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:44.377701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34996 (local address 127.10.114.67:41333)
0925 09:26:44.377923 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:44.377932 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:44.378552 (+ 620us) server_negotiation.cc:407] Connection header received
0925 09:26:44.378779 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:44.378784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:44.378856 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:44.378952 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:26:44.379385 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.380300 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.382255 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.382828 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.382980 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:44.383388 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:26:44.383490 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:26:44.383598 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:44.383756 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 09:26:44.383859 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 09:26:44.399937 12957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:44.393884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:43373 (local address 127.10.114.67:41333)
0925 09:26:44.394062 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:44.394068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:44.394701 (+ 633us) server_negotiation.cc:407] Connection header received
0925 09:26:44.394894 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:44.394900 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:44.394988 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:44.395080 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:26:44.395646 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.396555 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.398050 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.398519 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.398659 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:44.399105 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:26:44.399268 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:26:44.399373 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:44.399506 (+ 133us) server_negotiation.cc:299] Negotiation successful
0925 09:26:44.399576 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":44}
I20260925 09:26:44.400092 12995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:44.393956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:41333 (local address 127.10.114.67:43373)
0925 09:26:44.394580 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:44.394599 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:44.394782 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:44.395090 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:44.395094 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:44.395105 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:26:44.395464 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:44.395473 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.396739 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.396744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:44.397915 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:44.397925 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.398710 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.398723 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:44.398861 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:44.398909 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:26:44.399371 (+ 462us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:26:44.399382 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 09:26:44.399489 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 09:26:44.399610 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":315,"threads_started":1}
I20260925 09:26:44.410030 12616 raft_consensus.cc:1261] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025 [term 1 FOLLOWER]: Refusing update from remote peer 739baee21b8141b6a16b9b553c7c26b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:26:44.410765 12966 consensus_queue.cc:1035] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:26:44.412110 12750 raft_consensus.cc:1261] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851 [term 1 FOLLOWER]: Refusing update from remote peer 739baee21b8141b6a16b9b553c7c26b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:26:44.412842 12966 consensus_queue.cc:1035] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:26:44.424714 12985 mvcc.cc:205] Tried to move back new op lower bound from 7333185144454615040 to 7333185142951464960. Current Snapshot: MvccSnapshot[applied={T|T < 7333185144454615040}]
I20260925 09:26:44.429418 12984 mvcc.cc:205] Tried to move back new op lower bound from 7333185144454615040 to 7333185142951464960. Current Snapshot: MvccSnapshot[applied={T|T < 7333185144454615040}]
I20260925 09:26:44.483480 12452 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:26:44.483728 12452 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:26:44.487112 12452 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf: Sending DeleteTablet for 3 replicas of tablet 07b5c7fd2b0c4446b9d22281d991cc44
I20260925 09:26:44.488180 12591 tablet_service.cc:1514] Processing DeleteTablet for tablet 07b5c7fd2b0c4446b9d22281d991cc44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:26:44 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:57892
I20260925 09:26:44.488193 12730 tablet_service.cc:1514] Processing DeleteTablet for tablet 07b5c7fd2b0c4446b9d22281d991cc44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:26:44 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:45308
I20260925 09:26:44.488714 12869 tablet_service.cc:1514] Processing DeleteTablet for tablet 07b5c7fd2b0c4446b9d22281d991cc44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:26:44 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:34976
I20260925 09:26:44.491896 12452 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:26:44.492662 12452 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:26:44.497498 13002 tablet_replica.cc:330] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851: stopping tablet replica
I20260925 09:26:44.498102 13002 raft_consensus.cc:2229] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:26:44.499157 13003 tablet_replica.cc:330] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3: stopping tablet replica
I20260925 09:26:44.499584 13002 raft_consensus.cc:2258] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:26:44.500352 13003 raft_consensus.cc:2229] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:26:44.502159 13002 ts_tablet_manager.cc:1905] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:26:44.502806 12869 tablet_service.cc:1467] Processing CreateTablet for tablet d730904390884eaebde879041516b035 (DEFAULT_TABLE table=test-table [id=1e375af8d1a14fec867b5186c2309dae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:44.503334 12869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d730904390884eaebde879041516b035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:44.503755 12590 tablet_service.cc:1467] Processing CreateTablet for tablet d730904390884eaebde879041516b035 (DEFAULT_TABLE table=test-table [id=1e375af8d1a14fec867b5186c2309dae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:44.504326 12590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d730904390884eaebde879041516b035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:44.510205 13003 raft_consensus.cc:2258] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:26:44.511669 12960 tablet_bootstrap.cc:492] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025: Bootstrap starting.
I20260925 09:26:44.510509 12730 tablet_service.cc:1467] Processing CreateTablet for tablet d730904390884eaebde879041516b035 (DEFAULT_TABLE table=test-table [id=1e375af8d1a14fec867b5186c2309dae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:44.512492 12730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d730904390884eaebde879041516b035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:44.513882 12960 tablet_bootstrap.cc:654] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:44.513959 13001 tablet_replica.cc:330] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025: stopping tablet replica
I20260925 09:26:44.514362 13001 raft_consensus.cc:2229] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:26:44.514709 13001 raft_consensus.cc:2258] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:26:44.516028 12960 tablet_bootstrap.cc:492] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025: No bootstrap required, opened a new log
I20260925 09:26:44.516204 12960 ts_tablet_manager.cc:1397] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 09:26:44.516923 12960 raft_consensus.cc:348] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:44.517156 12960 raft_consensus.cc:374] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:44.517237 12960 raft_consensus.cc:729] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 146f2508850b4ffd81f52247de045025, State: Initialized, Role: FOLLOWER
I20260925 09:26:44.517511 12960 consensus_queue.cc:260] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:44.517693 13003 ts_tablet_manager.cc:1905] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:26:44.518143 13001 ts_tablet_manager.cc:1905] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:26:44.518748 12960 ts_tablet_manager.cc:1428] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 09:26:44.518823 12958 tablet_bootstrap.cc:492] T d730904390884eaebde879041516b035 P 04faf4158bb24bfb8333efe88b849851: Bootstrap starting.
I20260925 09:26:44.520367 12958 tablet_bootstrap.cc:654] T d730904390884eaebde879041516b035 P 04faf4158bb24bfb8333efe88b849851: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:44.523988 12959 tablet_bootstrap.cc:492] T d730904390884eaebde879041516b035 P 739baee21b8141b6a16b9b553c7c26b3: Bootstrap starting.
I20260925 09:26:44.525724 12959 tablet_bootstrap.cc:654] T d730904390884eaebde879041516b035 P 739baee21b8141b6a16b9b553c7c26b3: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:44.531560 13002 ts_tablet_manager.cc:1918] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:26:44.531813 13002 log.cc:1196] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/07b5c7fd2b0c4446b9d22281d991cc44
I20260925 09:26:44.532366 13002 ts_tablet_manager.cc:1939] T 07b5c7fd2b0c4446b9d22281d991cc44 P 04faf4158bb24bfb8333efe88b849851: Deleting consensus metadata
I20260925 09:26:44.533875 13001 ts_tablet_manager.cc:1918] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:26:44.534070 13001 log.cc:1196] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/07b5c7fd2b0c4446b9d22281d991cc44
I20260925 09:26:44.534099 13003 ts_tablet_manager.cc:1918] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:26:44.534348 13003 log.cc:1196] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/07b5c7fd2b0c4446b9d22281d991cc44
I20260925 09:26:44.534627 13001 ts_tablet_manager.cc:1939] T 07b5c7fd2b0c4446b9d22281d991cc44 P 146f2508850b4ffd81f52247de045025: Deleting consensus metadata
I20260925 09:26:44.534935 13003 ts_tablet_manager.cc:1939] T 07b5c7fd2b0c4446b9d22281d991cc44 P 739baee21b8141b6a16b9b553c7c26b3: Deleting consensus metadata
I20260925 09:26:44.536630 12441 catalog_manager.cc:4841] TS 739baee21b8141b6a16b9b553c7c26b3 (127.10.114.67:41333): tablet 07b5c7fd2b0c4446b9d22281d991cc44 (table test-table [id=826cf9b5a1024199838f15fa02f43053]) successfully deleted
I20260925 09:26:44.536864 12438 catalog_manager.cc:4841] TS 04faf4158bb24bfb8333efe88b849851 (127.10.114.66:39731): tablet 07b5c7fd2b0c4446b9d22281d991cc44 (table test-table [id=826cf9b5a1024199838f15fa02f43053]) successfully deleted
I20260925 09:26:44.537618 12959 tablet_bootstrap.cc:492] T d730904390884eaebde879041516b035 P 739baee21b8141b6a16b9b553c7c26b3: No bootstrap required, opened a new log
I20260925 09:26:44.537827 12959 ts_tablet_manager.cc:1397] T d730904390884eaebde879041516b035 P 739baee21b8141b6a16b9b553c7c26b3: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.004s
I20260925 09:26:44.538378 12958 tablet_bootstrap.cc:492] T d730904390884eaebde879041516b035 P 04faf4158bb24bfb8333efe88b849851: No bootstrap required, opened a new log
I20260925 09:26:44.538333 12440 catalog_manager.cc:4841] TS 146f2508850b4ffd81f52247de045025 (127.10.114.65:35275): tablet 07b5c7fd2b0c4446b9d22281d991cc44 (table test-table [id=826cf9b5a1024199838f15fa02f43053]) successfully deleted
I20260925 09:26:44.538547 12958 ts_tablet_manager.cc:1397] T d730904390884eaebde879041516b035 P 04faf4158bb24bfb8333efe88b849851: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.002s
I20260925 09:26:44.538734 12959 raft_consensus.cc:348] T d730904390884eaebde879041516b035 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:44.539012 12959 raft_consensus.cc:374] T d730904390884eaebde879041516b035 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:44.539124 12959 raft_consensus.cc:729] T d730904390884eaebde879041516b035 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 739baee21b8141b6a16b9b553c7c26b3, State: Initialized, Role: FOLLOWER
I20260925 09:26:44.539067 12958 raft_consensus.cc:348] T d730904390884eaebde879041516b035 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:44.539232 12958 raft_consensus.cc:374] T d730904390884eaebde879041516b035 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:44.539326 12958 raft_consensus.cc:729] T d730904390884eaebde879041516b035 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04faf4158bb24bfb8333efe88b849851, State: Initialized, Role: FOLLOWER
I20260925 09:26:44.539438 12959 consensus_queue.cc:260] T d730904390884eaebde879041516b035 P 739baee21b8141b6a16b9b553c7c26b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:44.539577 12958 consensus_queue.cc:260] T d730904390884eaebde879041516b035 P 04faf4158bb24bfb8333efe88b849851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:44.540215 12958 ts_tablet_manager.cc:1428] T d730904390884eaebde879041516b035 P 04faf4158bb24bfb8333efe88b849851: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 09:26:44.540602 12959 ts_tablet_manager.cc:1428] T d730904390884eaebde879041516b035 P 739baee21b8141b6a16b9b553c7c26b3: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260925 09:26:44.562885 12964 raft_consensus.cc:482] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:26:44.563055 12964 raft_consensus.cc:504] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:44.564574 12964 leader_election.cc:288] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04faf4158bb24bfb8333efe88b849851 (127.10.114.66:39731), 739baee21b8141b6a16b9b553c7c26b3 (127.10.114.67:41333)
I20260925 09:26:44.587545 13009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:44.565742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:43715 (local address 127.10.114.66:39731)
0925 09:26:44.579361 (+ 13619us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:44.579369 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:44.580320 (+ 951us) server_negotiation.cc:407] Connection header received
0925 09:26:44.580437 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:44.580442 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:44.580508 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:44.580659 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:44.582071 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.583241 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.585737 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.586520 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.586729 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:44.586813 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:44.587081 (+ 268us) server_negotiation.cc:299] Negotiation successful
0925 09:26:44.587214 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13447,"thread_start_us":13621,"threads_started":1}
I20260925 09:26:44.587594 13008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:44.564814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:39731 (local address 127.10.114.65:43715)
0925 09:26:44.580069 (+ 15255us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:44.580087 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:44.580217 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:44.580853 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:44.580858 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:44.580881 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:44.581716 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:44.581733 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.583401 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.583409 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:44.585588 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:44.585598 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.586719 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.586726 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:44.586902 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:44.586926 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 09:26:44.587045 (+ 119us) client_negotiation.cc:240] Negotiation successful
0925 09:26:44.587192 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15126,"mutex_wait_us":370,"thread_start_us":14894,"threads_started":1}
I20260925 09:26:44.587766 12957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:44.565308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:60795 (local address 127.10.114.67:41333)
0925 09:26:44.565542 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:44.565552 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:44.578401 (+ 12849us) server_negotiation.cc:407] Connection header received
0925 09:26:44.580135 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:44.580143 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:44.580230 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:44.580308 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:44.581368 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.582605 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.585328 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.586622 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.586765 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:44.586856 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:44.587389 (+ 533us) server_negotiation.cc:299] Negotiation successful
0925 09:26:44.587511 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260925 09:26:44.588171 12750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d730904390884eaebde879041516b035" candidate_uuid: "146f2508850b4ffd81f52247de045025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04faf4158bb24bfb8333efe88b849851" is_pre_election: true
I20260925 09:26:44.588435 12750 raft_consensus.cc:2454] T d730904390884eaebde879041516b035 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 146f2508850b4ffd81f52247de045025 in term 0.
I20260925 09:26:44.588630 13007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:44.564814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:41333 (local address 127.10.114.65:60795)
0925 09:26:44.578267 (+ 13453us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:44.578292 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:44.579601 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:44.580610 (+ 1009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:44.580615 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:44.580661 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:44.581195 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:44.581208 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.582801 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.582807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:44.585143 (+ 2336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:44.585160 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.587026 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.587032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:44.587211 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:44.587250 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 09:26:44.588162 (+ 912us) client_negotiation.cc:240] Negotiation successful
0925 09:26:44.588306 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13293,"thread_start_us":13163,"threads_started":1}
I20260925 09:26:44.589074 12541 leader_election.cc:302] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04faf4158bb24bfb8333efe88b849851, 146f2508850b4ffd81f52247de045025; no voters:
I20260925 09:26:44.589546 12964 raft_consensus.cc:2781] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:26:44.589519 12893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d730904390884eaebde879041516b035" candidate_uuid: "146f2508850b4ffd81f52247de045025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "739baee21b8141b6a16b9b553c7c26b3" is_pre_election: true
I20260925 09:26:44.589706 12964 raft_consensus.cc:482] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:26:44.589841 12964 raft_consensus.cc:3037] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:44.589949 12893 raft_consensus.cc:2454] T d730904390884eaebde879041516b035 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 146f2508850b4ffd81f52247de045025 in term 0.
I20260925 09:26:44.592500 12964 raft_consensus.cc:504] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:44.593165 12964 leader_election.cc:288] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [CANDIDATE]: Term 1 election: Requested vote from peers 04faf4158bb24bfb8333efe88b849851 (127.10.114.66:39731), 739baee21b8141b6a16b9b553c7c26b3 (127.10.114.67:41333)
I20260925 09:26:44.593554 12750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d730904390884eaebde879041516b035" candidate_uuid: "146f2508850b4ffd81f52247de045025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04faf4158bb24bfb8333efe88b849851"
I20260925 09:26:44.593621 12893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d730904390884eaebde879041516b035" candidate_uuid: "146f2508850b4ffd81f52247de045025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "739baee21b8141b6a16b9b553c7c26b3"
I20260925 09:26:44.593779 12750 raft_consensus.cc:3037] T d730904390884eaebde879041516b035 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:44.593816 12893 raft_consensus.cc:3037] T d730904390884eaebde879041516b035 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:44.596781 12750 raft_consensus.cc:2454] T d730904390884eaebde879041516b035 P 04faf4158bb24bfb8333efe88b849851 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 146f2508850b4ffd81f52247de045025 in term 1.
I20260925 09:26:44.597124 12893 raft_consensus.cc:2454] T d730904390884eaebde879041516b035 P 739baee21b8141b6a16b9b553c7c26b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 146f2508850b4ffd81f52247de045025 in term 1.
I20260925 09:26:44.597323 12541 leader_election.cc:302] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04faf4158bb24bfb8333efe88b849851, 146f2508850b4ffd81f52247de045025; no voters:
I20260925 09:26:44.597630 12964 raft_consensus.cc:2781] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:26:44.598271 12964 raft_consensus.cc:686] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [term 1 LEADER]: Becoming Leader. State: Replica: 146f2508850b4ffd81f52247de045025, State: Running, Role: LEADER
I20260925 09:26:44.598719 12964 consensus_queue.cc:237] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:44.602306 12453 catalog_manager.cc:5495] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 reported cstate change: term changed from 0 to 1, leader changed from <none> to 146f2508850b4ffd81f52247de045025 (127.10.114.65). New cstate: current_term: 1 leader_uuid: "146f2508850b4ffd81f52247de045025" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } health_report { overall_health: UNKNOWN } } }
I20260925 09:26:44.637209 13012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:44.628220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57896 (local address 127.10.114.65:35275)
0925 09:26:44.631574 (+ 3354us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:44.631578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:44.631595 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:26:44.631663 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:44.631667 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:44.631735 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:44.631918 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:26:44.632401 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.633182 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.635014 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.635433 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.635527 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:44.635995 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:26:44.636533 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:26:44.636634 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:44.636833 (+ 199us) server_negotiation.cc:299] Negotiation successful
0925 09:26:44.636916 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3266,"thread_start_us":3096,"threads_started":1}
I20260925 09:26:44.652750 13015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:44.646833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:41333 (local address 127.10.114.65:55513)
0925 09:26:44.647270 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:44.647289 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:44.647383 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:44.647710 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:44.647713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:44.647721 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:26:44.647952 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:44.647958 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.649294 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.649298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:44.650560 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:44.650572 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.651551 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.651556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:44.651702 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:44.651750 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:26:44.652278 (+ 528us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:26:44.652289 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 09:26:44.652364 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 09:26:44.652455 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":230,"threads_started":1}
I20260925 09:26:44.652783 12957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:44.646788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:55513 (local address 127.10.114.67:41333)
0925 09:26:44.646978 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:44.646985 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:44.647370 (+ 385us) server_negotiation.cc:407] Connection header received
0925 09:26:44.647495 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:44.647500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:44.647592 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:44.647684 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:26:44.648083 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.649111 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.650720 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.651361 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.651494 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:44.652013 (+ 519us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:26:44.652137 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:26:44.652258 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:44.652413 (+ 155us) server_negotiation.cc:299] Negotiation successful
0925 09:26:44.652485 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 09:26:44.661556 12750 raft_consensus.cc:1261] T d730904390884eaebde879041516b035 P 04faf4158bb24bfb8333efe88b849851 [term 1 FOLLOWER]: Refusing update from remote peer 146f2508850b4ffd81f52247de045025: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:26:44.661829 12893 raft_consensus.cc:1261] T d730904390884eaebde879041516b035 P 739baee21b8141b6a16b9b553c7c26b3 [term 1 FOLLOWER]: Refusing update from remote peer 146f2508850b4ffd81f52247de045025: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:26:44.662245 13010 consensus_queue.cc:1035] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:26:44.663178 13010 consensus_queue.cc:1035] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [LEADER]: Connected to new peer: Peer: permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:26:44.695807 12995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:44.689320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:35275 (local address 127.10.114.67:35589)
0925 09:26:44.689689 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:44.689708 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:44.689819 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:44.690586 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:44.690591 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:44.690603 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:26:44.690792 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:44.690803 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.691957 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.691966 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:44.693551 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:44.693566 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.694521 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.694527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:44.694728 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:44.694785 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:26:44.695297 (+ 512us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:26:44.695312 (+ 15us) client_negotiation.cc:769] Sending connection context
0925 09:26:44.695405 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 09:26:44.695504 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234}
I20260925 09:26:44.696266 13012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:44.690002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:35589 (local address 127.10.114.65:35275)
0925 09:26:44.690156 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:44.690163 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:44.690180 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:26:44.690247 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:44.690252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:44.690329 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:44.690416 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:26:44.690965 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.691824 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.693712 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:44.694376 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:44.694474 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:44.694987 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:26:44.695135 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:26:44.695248 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:44.695447 (+ 199us) server_negotiation.cc:299] Negotiation successful
0925 09:26:44.695526 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260925 09:26:44.697284 12985 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 09:26:46.301944 12453 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:26:46.302189 12453 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:26:46.305637 12453 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf: Sending DeleteTablet for 3 replicas of tablet d730904390884eaebde879041516b035
I20260925 09:26:46.306357 12591 tablet_service.cc:1514] Processing DeleteTablet for tablet d730904390884eaebde879041516b035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:26:46 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:57892
I20260925 09:26:46.306464 12869 tablet_service.cc:1514] Processing DeleteTablet for tablet d730904390884eaebde879041516b035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:26:46 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:34976
I20260925 09:26:46.306489 12730 tablet_service.cc:1514] Processing DeleteTablet for tablet d730904390884eaebde879041516b035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:26:46 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:45308
I20260925 09:26:46.318912 13042 tablet_replica.cc:330] T d730904390884eaebde879041516b035 P 739baee21b8141b6a16b9b553c7c26b3: stopping tablet replica
I20260925 09:26:46.318917 13041 tablet_replica.cc:330] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025: stopping tablet replica
I20260925 09:26:46.319106 13042 raft_consensus.cc:2229] T d730904390884eaebde879041516b035 P 739baee21b8141b6a16b9b553c7c26b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:26:46.319175 13041 raft_consensus.cc:2229] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:26:46.319339 13042 raft_consensus.cc:2258] T d730904390884eaebde879041516b035 P 739baee21b8141b6a16b9b553c7c26b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:26:46.319886 13041 raft_consensus.cc:2258] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:26:46.320209 13042 ts_tablet_manager.cc:1905] T d730904390884eaebde879041516b035 P 739baee21b8141b6a16b9b553c7c26b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:26:46.321225 13041 ts_tablet_manager.cc:1905] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:26:46.321396 13043 tablet_replica.cc:330] T d730904390884eaebde879041516b035 P 04faf4158bb24bfb8333efe88b849851: stopping tablet replica
I20260925 09:26:46.321565 13043 raft_consensus.cc:2229] T d730904390884eaebde879041516b035 P 04faf4158bb24bfb8333efe88b849851 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:26:46.321776 13043 raft_consensus.cc:2258] T d730904390884eaebde879041516b035 P 04faf4158bb24bfb8333efe88b849851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:26:46.322587 13043 ts_tablet_manager.cc:1905] T d730904390884eaebde879041516b035 P 04faf4158bb24bfb8333efe88b849851: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:26:46.326037 13041 ts_tablet_manager.cc:1918] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 09:26:46.326180 13041 log.cc:1196] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/d730904390884eaebde879041516b035
I20260925 09:26:46.326432 13042 ts_tablet_manager.cc:1918] T d730904390884eaebde879041516b035 P 739baee21b8141b6a16b9b553c7c26b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 09:26:46.326663 13042 log.cc:1196] T d730904390884eaebde879041516b035 P 739baee21b8141b6a16b9b553c7c26b3: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/d730904390884eaebde879041516b035
I20260925 09:26:46.326761 13041 ts_tablet_manager.cc:1939] T d730904390884eaebde879041516b035 P 146f2508850b4ffd81f52247de045025: Deleting consensus metadata
I20260925 09:26:46.327121 13042 ts_tablet_manager.cc:1939] T d730904390884eaebde879041516b035 P 739baee21b8141b6a16b9b553c7c26b3: Deleting consensus metadata
I20260925 09:26:46.328012 12440 catalog_manager.cc:4841] TS 146f2508850b4ffd81f52247de045025 (127.10.114.65:35275): tablet d730904390884eaebde879041516b035 (table test-table [id=1e375af8d1a14fec867b5186c2309dae]) successfully deleted
I20260925 09:26:46.328097 12441 catalog_manager.cc:4841] TS 739baee21b8141b6a16b9b553c7c26b3 (127.10.114.67:41333): tablet d730904390884eaebde879041516b035 (table test-table [id=1e375af8d1a14fec867b5186c2309dae]) successfully deleted
I20260925 09:26:46.328679 13043 ts_tablet_manager.cc:1918] T d730904390884eaebde879041516b035 P 04faf4158bb24bfb8333efe88b849851: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 09:26:46.328773 13043 log.cc:1196] T d730904390884eaebde879041516b035 P 04faf4158bb24bfb8333efe88b849851: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/d730904390884eaebde879041516b035
I20260925 09:26:46.329134 13043 ts_tablet_manager.cc:1939] T d730904390884eaebde879041516b035 P 04faf4158bb24bfb8333efe88b849851: Deleting consensus metadata
I20260925 09:26:46.329938 12438 catalog_manager.cc:4841] TS 04faf4158bb24bfb8333efe88b849851 (127.10.114.66:39731): tablet d730904390884eaebde879041516b035 (table test-table [id=1e375af8d1a14fec867b5186c2309dae]) successfully deleted
Sep 25 09:26:46 dist-test-slave-9pj8 krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328406, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 09:26:46 dist-test-slave-9pj8 krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328406, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:46.367060 13051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:46.353243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50726 (local address 127.10.114.126:42763)
0925 09:26:46.353843 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:46.353847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:46.353868 (+ 21us) server_negotiation.cc:407] Connection header received
0925 09:26:46.353991 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:46.354013 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:46.354077 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:46.354173 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:46.355518 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.356369 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.357553 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.357950 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.358059 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:46.362073 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:46.362078 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:46.362081 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:46.362116 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:46.364611 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:46.365374 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:46.365378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:46.365380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:46.365500 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:46.365864 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:46.365875 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:46.365877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:46.366120 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:46.366298 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:46.366560 (+ 262us) server_negotiation.cc:299] Negotiation successful
0925 09:26:46.366722 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":291,"threads_started":1}
I20260925 09:26:46.370137 12453 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:26:46.370712 12453 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:26:46.377135 12591 tablet_service.cc:1467] Processing CreateTablet for tablet 21b6efd833614b079ab631891d5ee319 (DEFAULT_TABLE table=test-table [id=df12142e82ff4d44b430c8e130f210cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:46.377427 12869 tablet_service.cc:1467] Processing CreateTablet for tablet 21b6efd833614b079ab631891d5ee319 (DEFAULT_TABLE table=test-table [id=df12142e82ff4d44b430c8e130f210cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:46.377458 12730 tablet_service.cc:1467] Processing CreateTablet for tablet 21b6efd833614b079ab631891d5ee319 (DEFAULT_TABLE table=test-table [id=df12142e82ff4d44b430c8e130f210cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:46.377557 12591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21b6efd833614b079ab631891d5ee319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:46.377811 12730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21b6efd833614b079ab631891d5ee319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:46.377825 12869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21b6efd833614b079ab631891d5ee319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:46.383339 13052 tablet_bootstrap.cc:492] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025: Bootstrap starting.
I20260925 09:26:46.384542 13053 tablet_bootstrap.cc:492] T 21b6efd833614b079ab631891d5ee319 P 04faf4158bb24bfb8333efe88b849851: Bootstrap starting.
I20260925 09:26:46.385197 13054 tablet_bootstrap.cc:492] T 21b6efd833614b079ab631891d5ee319 P 739baee21b8141b6a16b9b553c7c26b3: Bootstrap starting.
I20260925 09:26:46.385627 13052 tablet_bootstrap.cc:654] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:46.386340 13053 tablet_bootstrap.cc:654] T 21b6efd833614b079ab631891d5ee319 P 04faf4158bb24bfb8333efe88b849851: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:46.386734 13054 tablet_bootstrap.cc:654] T 21b6efd833614b079ab631891d5ee319 P 739baee21b8141b6a16b9b553c7c26b3: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:46.388581 13053 tablet_bootstrap.cc:492] T 21b6efd833614b079ab631891d5ee319 P 04faf4158bb24bfb8333efe88b849851: No bootstrap required, opened a new log
I20260925 09:26:46.388624 13052 tablet_bootstrap.cc:492] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025: No bootstrap required, opened a new log
I20260925 09:26:46.388757 13053 ts_tablet_manager.cc:1397] T 21b6efd833614b079ab631891d5ee319 P 04faf4158bb24bfb8333efe88b849851: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260925 09:26:46.388770 13052 ts_tablet_manager.cc:1397] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260925 09:26:46.389329 13054 tablet_bootstrap.cc:492] T 21b6efd833614b079ab631891d5ee319 P 739baee21b8141b6a16b9b553c7c26b3: No bootstrap required, opened a new log
I20260925 09:26:46.389513 13054 ts_tablet_manager.cc:1397] T 21b6efd833614b079ab631891d5ee319 P 739baee21b8141b6a16b9b553c7c26b3: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20260925 09:26:46.389566 13052 raft_consensus.cc:348] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:46.389648 13053 raft_consensus.cc:348] T 21b6efd833614b079ab631891d5ee319 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:46.389842 13052 raft_consensus.cc:374] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:46.389896 13053 raft_consensus.cc:374] T 21b6efd833614b079ab631891d5ee319 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:46.389940 13052 raft_consensus.cc:729] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 146f2508850b4ffd81f52247de045025, State: Initialized, Role: FOLLOWER
I20260925 09:26:46.389972 13053 raft_consensus.cc:729] T 21b6efd833614b079ab631891d5ee319 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04faf4158bb24bfb8333efe88b849851, State: Initialized, Role: FOLLOWER
I20260925 09:26:46.390170 13052 consensus_queue.cc:260] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:46.390172 13053 consensus_queue.cc:260] T 21b6efd833614b079ab631891d5ee319 P 04faf4158bb24bfb8333efe88b849851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:46.390741 13052 ts_tablet_manager.cc:1428] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 09:26:46.390933 13053 ts_tablet_manager.cc:1428] T 21b6efd833614b079ab631891d5ee319 P 04faf4158bb24bfb8333efe88b849851: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 09:26:46.390290 13054 raft_consensus.cc:348] T 21b6efd833614b079ab631891d5ee319 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:46.391486 13054 raft_consensus.cc:374] T 21b6efd833614b079ab631891d5ee319 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:46.391593 13054 raft_consensus.cc:729] T 21b6efd833614b079ab631891d5ee319 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 739baee21b8141b6a16b9b553c7c26b3, State: Initialized, Role: FOLLOWER
I20260925 09:26:46.391835 13054 consensus_queue.cc:260] T 21b6efd833614b079ab631891d5ee319 P 739baee21b8141b6a16b9b553c7c26b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:46.392450 13054 ts_tablet_manager.cc:1428] T 21b6efd833614b079ab631891d5ee319 P 739baee21b8141b6a16b9b553c7c26b3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 09:26:46.468796 13031 raft_consensus.cc:482] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:26:46.469030 13031 raft_consensus.cc:504] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:46.469741 13031 leader_election.cc:288] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04faf4158bb24bfb8333efe88b849851 (127.10.114.66:39731), 739baee21b8141b6a16b9b553c7c26b3 (127.10.114.67:41333)
I20260925 09:26:46.470335 12750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21b6efd833614b079ab631891d5ee319" candidate_uuid: "146f2508850b4ffd81f52247de045025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04faf4158bb24bfb8333efe88b849851" is_pre_election: true
I20260925 09:26:46.470335 12893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21b6efd833614b079ab631891d5ee319" candidate_uuid: "146f2508850b4ffd81f52247de045025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "739baee21b8141b6a16b9b553c7c26b3" is_pre_election: true
I20260925 09:26:46.470590 12893 raft_consensus.cc:2454] T 21b6efd833614b079ab631891d5ee319 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 146f2508850b4ffd81f52247de045025 in term 0.
I20260925 09:26:46.470590 12750 raft_consensus.cc:2454] T 21b6efd833614b079ab631891d5ee319 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 146f2508850b4ffd81f52247de045025 in term 0.
I20260925 09:26:46.471053 12541 leader_election.cc:302] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04faf4158bb24bfb8333efe88b849851, 146f2508850b4ffd81f52247de045025; no voters:
I20260925 09:26:46.471335 13031 raft_consensus.cc:2781] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:26:46.471458 13031 raft_consensus.cc:482] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:26:46.471562 13031 raft_consensus.cc:3037] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:46.473910 13031 raft_consensus.cc:504] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:46.474395 13031 leader_election.cc:288] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [CANDIDATE]: Term 1 election: Requested vote from peers 04faf4158bb24bfb8333efe88b849851 (127.10.114.66:39731), 739baee21b8141b6a16b9b553c7c26b3 (127.10.114.67:41333)
I20260925 09:26:46.474829 12893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21b6efd833614b079ab631891d5ee319" candidate_uuid: "146f2508850b4ffd81f52247de045025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "739baee21b8141b6a16b9b553c7c26b3"
I20260925 09:26:46.474852 12750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21b6efd833614b079ab631891d5ee319" candidate_uuid: "146f2508850b4ffd81f52247de045025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04faf4158bb24bfb8333efe88b849851"
I20260925 09:26:46.474995 12893 raft_consensus.cc:3037] T 21b6efd833614b079ab631891d5ee319 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:46.475080 12750 raft_consensus.cc:3037] T 21b6efd833614b079ab631891d5ee319 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:46.476961 12893 raft_consensus.cc:2454] T 21b6efd833614b079ab631891d5ee319 P 739baee21b8141b6a16b9b553c7c26b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 146f2508850b4ffd81f52247de045025 in term 1.
I20260925 09:26:46.477103 12750 raft_consensus.cc:2454] T 21b6efd833614b079ab631891d5ee319 P 04faf4158bb24bfb8333efe88b849851 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 146f2508850b4ffd81f52247de045025 in term 1.
I20260925 09:26:46.477365 12544 leader_election.cc:302] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 146f2508850b4ffd81f52247de045025, 739baee21b8141b6a16b9b553c7c26b3; no voters:
I20260925 09:26:46.477605 13031 raft_consensus.cc:2781] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:26:46.477757 13031 raft_consensus.cc:686] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [term 1 LEADER]: Becoming Leader. State: Replica: 146f2508850b4ffd81f52247de045025, State: Running, Role: LEADER
I20260925 09:26:46.478008 13031 consensus_queue.cc:237] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:46.480685 12451 catalog_manager.cc:5495] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 reported cstate change: term changed from 0 to 1, leader changed from <none> to 146f2508850b4ffd81f52247de045025 (127.10.114.65). New cstate: current_term: 1 leader_uuid: "146f2508850b4ffd81f52247de045025" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } health_report { overall_health: UNKNOWN } } }
I20260925 09:26:46.521355 13061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:46.516868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57902 (local address 127.10.114.65:35275)
0925 09:26:46.517406 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:46.517410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:46.517427 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:26:46.517472 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:46.517475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:46.517530 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:46.517624 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:26:46.517957 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.518705 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.519960 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.520354 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.520442 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:46.520795 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:26:46.520888 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:26:46.520949 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:46.521071 (+ 122us) server_negotiation.cc:299] Negotiation successful
0925 09:26:46.521122 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":292,"threads_started":1}
I20260925 09:26:46.529589 12893 raft_consensus.cc:1261] T 21b6efd833614b079ab631891d5ee319 P 739baee21b8141b6a16b9b553c7c26b3 [term 1 FOLLOWER]: Refusing update from remote peer 146f2508850b4ffd81f52247de045025: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:26:46.529633 12750 raft_consensus.cc:1261] T 21b6efd833614b079ab631891d5ee319 P 04faf4158bb24bfb8333efe88b849851 [term 1 FOLLOWER]: Refusing update from remote peer 146f2508850b4ffd81f52247de045025: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:26:46.530071 13031 consensus_queue.cc:1035] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [LEADER]: Connected to new peer: Peer: permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:26:46.530319 13019 consensus_queue.cc:1035] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:26:46.569661 12451 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:26:46.569875 12451 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:26:46.572540 12451 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf: Sending DeleteTablet for 3 replicas of tablet 21b6efd833614b079ab631891d5ee319
I20260925 09:26:46.573243 12730 tablet_service.cc:1514] Processing DeleteTablet for tablet 21b6efd833614b079ab631891d5ee319 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:26:46 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:45308
I20260925 09:26:46.573221 12591 tablet_service.cc:1514] Processing DeleteTablet for tablet 21b6efd833614b079ab631891d5ee319 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:26:46 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:57892
I20260925 09:26:46.573285 12869 tablet_service.cc:1514] Processing DeleteTablet for tablet 21b6efd833614b079ab631891d5ee319 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:26:46 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:34976
I20260925 09:26:46.573540 13043 tablet_replica.cc:330] T 21b6efd833614b079ab631891d5ee319 P 04faf4158bb24bfb8333efe88b849851: stopping tablet replica
I20260925 09:26:46.573539 13041 tablet_replica.cc:330] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025: stopping tablet replica
I20260925 09:26:46.573539 13042 tablet_replica.cc:330] T 21b6efd833614b079ab631891d5ee319 P 739baee21b8141b6a16b9b553c7c26b3: stopping tablet replica
I20260925 09:26:46.573714 13042 raft_consensus.cc:2229] T 21b6efd833614b079ab631891d5ee319 P 739baee21b8141b6a16b9b553c7c26b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:26:46.573742 13041 raft_consensus.cc:2229] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:26:46.573721 13043 raft_consensus.cc:2229] T 21b6efd833614b079ab631891d5ee319 P 04faf4158bb24bfb8333efe88b849851 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:26:46.573904 13043 raft_consensus.cc:2258] T 21b6efd833614b079ab631891d5ee319 P 04faf4158bb24bfb8333efe88b849851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:26:46.573905 13042 raft_consensus.cc:2258] T 21b6efd833614b079ab631891d5ee319 P 739baee21b8141b6a16b9b553c7c26b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:26:46.573999 13041 raft_consensus.cc:2258] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:26:46.574855 13042 ts_tablet_manager.cc:1905] T 21b6efd833614b079ab631891d5ee319 P 739baee21b8141b6a16b9b553c7c26b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:26:46.574900 13043 ts_tablet_manager.cc:1905] T 21b6efd833614b079ab631891d5ee319 P 04faf4158bb24bfb8333efe88b849851: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:26:46.575057 13041 ts_tablet_manager.cc:1905] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:26:46.579166 13042 ts_tablet_manager.cc:1918] T 21b6efd833614b079ab631891d5ee319 P 739baee21b8141b6a16b9b553c7c26b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:26:46.579314 13042 log.cc:1196] T 21b6efd833614b079ab631891d5ee319 P 739baee21b8141b6a16b9b553c7c26b3: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/21b6efd833614b079ab631891d5ee319
I20260925 09:26:46.579990 13042 ts_tablet_manager.cc:1939] T 21b6efd833614b079ab631891d5ee319 P 739baee21b8141b6a16b9b553c7c26b3: Deleting consensus metadata
I20260925 09:26:46.580174 12451 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:26:46.581010 12451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:26:46.581259 12441 catalog_manager.cc:4841] TS 739baee21b8141b6a16b9b553c7c26b3 (127.10.114.67:41333): tablet 21b6efd833614b079ab631891d5ee319 (table test-table [id=df12142e82ff4d44b430c8e130f210cf]) successfully deleted
I20260925 09:26:46.581523 13041 ts_tablet_manager.cc:1918] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:26:46.581655 13043 ts_tablet_manager.cc:1918] T 21b6efd833614b079ab631891d5ee319 P 04faf4158bb24bfb8333efe88b849851: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:26:46.581678 13041 log.cc:1196] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/21b6efd833614b079ab631891d5ee319
I20260925 09:26:46.581794 13043 log.cc:1196] T 21b6efd833614b079ab631891d5ee319 P 04faf4158bb24bfb8333efe88b849851: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/21b6efd833614b079ab631891d5ee319
I20260925 09:26:46.582124 13041 ts_tablet_manager.cc:1939] T 21b6efd833614b079ab631891d5ee319 P 146f2508850b4ffd81f52247de045025: Deleting consensus metadata
I20260925 09:26:46.582160 13043 ts_tablet_manager.cc:1939] T 21b6efd833614b079ab631891d5ee319 P 04faf4158bb24bfb8333efe88b849851: Deleting consensus metadata
I20260925 09:26:46.583024 12440 catalog_manager.cc:4841] TS 146f2508850b4ffd81f52247de045025 (127.10.114.65:35275): tablet 21b6efd833614b079ab631891d5ee319 (table test-table [id=df12142e82ff4d44b430c8e130f210cf]) successfully deleted
I20260925 09:26:46.583148 12438 catalog_manager.cc:4841] TS 04faf4158bb24bfb8333efe88b849851 (127.10.114.66:39731): tablet 21b6efd833614b079ab631891d5ee319 (table test-table [id=df12142e82ff4d44b430c8e130f210cf]) successfully deleted
I20260925 09:26:46.586905 12591 tablet_service.cc:1467] Processing CreateTablet for tablet d9a11271fd7449c49aa14e49933158f6 (DEFAULT_TABLE table=test-table [id=afef04e9663644eb85fe47be77656c82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:46.586982 12730 tablet_service.cc:1467] Processing CreateTablet for tablet d9a11271fd7449c49aa14e49933158f6 (DEFAULT_TABLE table=test-table [id=afef04e9663644eb85fe47be77656c82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:46.587260 12591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9a11271fd7449c49aa14e49933158f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:46.587271 12730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9a11271fd7449c49aa14e49933158f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:46.591127 13052 tablet_bootstrap.cc:492] T d9a11271fd7449c49aa14e49933158f6 P 146f2508850b4ffd81f52247de045025: Bootstrap starting.
I20260925 09:26:46.591174 13053 tablet_bootstrap.cc:492] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851: Bootstrap starting.
I20260925 09:26:46.591609 12869 tablet_service.cc:1467] Processing CreateTablet for tablet d9a11271fd7449c49aa14e49933158f6 (DEFAULT_TABLE table=test-table [id=afef04e9663644eb85fe47be77656c82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:26:46.592024 12869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9a11271fd7449c49aa14e49933158f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:46.592672 13053 tablet_bootstrap.cc:654] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:46.592700 13052 tablet_bootstrap.cc:654] T d9a11271fd7449c49aa14e49933158f6 P 146f2508850b4ffd81f52247de045025: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:46.594671 13052 tablet_bootstrap.cc:492] T d9a11271fd7449c49aa14e49933158f6 P 146f2508850b4ffd81f52247de045025: No bootstrap required, opened a new log
I20260925 09:26:46.594857 13052 ts_tablet_manager.cc:1397] T d9a11271fd7449c49aa14e49933158f6 P 146f2508850b4ffd81f52247de045025: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260925 09:26:46.595021 13053 tablet_bootstrap.cc:492] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851: No bootstrap required, opened a new log
I20260925 09:26:46.595177 13053 ts_tablet_manager.cc:1397] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260925 09:26:46.595496 13052 raft_consensus.cc:348] T d9a11271fd7449c49aa14e49933158f6 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:46.595695 13052 raft_consensus.cc:374] T d9a11271fd7449c49aa14e49933158f6 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:46.595818 13052 raft_consensus.cc:729] T d9a11271fd7449c49aa14e49933158f6 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 146f2508850b4ffd81f52247de045025, State: Initialized, Role: FOLLOWER
I20260925 09:26:46.595916 13053 raft_consensus.cc:348] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:46.596215 13053 raft_consensus.cc:374] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:46.596149 13052 consensus_queue.cc:260] T d9a11271fd7449c49aa14e49933158f6 P 146f2508850b4ffd81f52247de045025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:46.596287 13053 raft_consensus.cc:729] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04faf4158bb24bfb8333efe88b849851, State: Initialized, Role: FOLLOWER
I20260925 09:26:46.596767 13052 ts_tablet_manager.cc:1428] T d9a11271fd7449c49aa14e49933158f6 P 146f2508850b4ffd81f52247de045025: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 09:26:46.596503 13053 consensus_queue.cc:260] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:46.597549 13053 ts_tablet_manager.cc:1428] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 09:26:46.598353 13054 tablet_bootstrap.cc:492] T d9a11271fd7449c49aa14e49933158f6 P 739baee21b8141b6a16b9b553c7c26b3: Bootstrap starting.
I20260925 09:26:46.599843 13054 tablet_bootstrap.cc:654] T d9a11271fd7449c49aa14e49933158f6 P 739baee21b8141b6a16b9b553c7c26b3: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:46.601897 13054 tablet_bootstrap.cc:492] T d9a11271fd7449c49aa14e49933158f6 P 739baee21b8141b6a16b9b553c7c26b3: No bootstrap required, opened a new log
I20260925 09:26:46.602056 13054 ts_tablet_manager.cc:1397] T d9a11271fd7449c49aa14e49933158f6 P 739baee21b8141b6a16b9b553c7c26b3: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260925 09:26:46.603025 13054 raft_consensus.cc:348] T d9a11271fd7449c49aa14e49933158f6 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:46.603227 13054 raft_consensus.cc:374] T d9a11271fd7449c49aa14e49933158f6 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:46.603329 13054 raft_consensus.cc:729] T d9a11271fd7449c49aa14e49933158f6 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 739baee21b8141b6a16b9b553c7c26b3, State: Initialized, Role: FOLLOWER
I20260925 09:26:46.603525 13054 consensus_queue.cc:260] T d9a11271fd7449c49aa14e49933158f6 P 739baee21b8141b6a16b9b553c7c26b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:46.604018 13054 ts_tablet_manager.cc:1428] T d9a11271fd7449c49aa14e49933158f6 P 739baee21b8141b6a16b9b553c7c26b3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 09:26:46.639058 13058 raft_consensus.cc:482] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:26:46.639276 13058 raft_consensus.cc:504] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:46.640731 13058 leader_election.cc:288] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 146f2508850b4ffd81f52247de045025 (127.10.114.65:35275), 739baee21b8141b6a16b9b553c7c26b3 (127.10.114.67:41333)
I20260925 09:26:46.647728 13061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:46.640982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:39883 (local address 127.10.114.65:35275)
0925 09:26:46.641163 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:46.641168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:46.641766 (+ 598us) server_negotiation.cc:407] Connection header received
0925 09:26:46.642073 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:46.642078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:46.642167 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:46.642266 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:46.643100 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.643872 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.645690 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.646542 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.646650 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:46.646699 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:46.647358 (+ 659us) server_negotiation.cc:299] Negotiation successful
0925 09:26:46.647504 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:26:46.647832 13068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:46.640898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:35275 (local address 127.10.114.66:39883)
0925 09:26:46.641640 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:46.641663 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:46.641896 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:46.642460 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:46.642465 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:46.642491 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:46.642854 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:46.642872 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.643998 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.644002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:46.645522 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:46.645532 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.646989 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.646995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:46.647169 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:46.647205 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 09:26:46.647321 (+ 116us) client_negotiation.cc:240] Negotiation successful
0925 09:26:46.647450 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":1,"thread_start_us":437,"threads_started":1}
I20260925 09:26:46.648183 13070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:46.641155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:41333 (local address 127.10.114.66:35643)
0925 09:26:46.642165 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:46.642178 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:46.642331 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:46.642727 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:46.642737 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:46.642762 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:26:46.643174 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:46.643183 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.644220 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.644226 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:46.645731 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:46.645744 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.647470 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.647476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:46.647630 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:46.647656 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 09:26:46.647748 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 09:26:46.647878 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"thread_start_us":701,"threads_started":1}
I20260925 09:26:46.648212 13069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:46.641164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:35643 (local address 127.10.114.67:41333)
0925 09:26:46.641895 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:46.641903 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:46.642454 (+ 551us) server_negotiation.cc:407] Connection header received
0925 09:26:46.642540 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:46.642544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:46.642597 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:46.642695 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:26:46.643327 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.644102 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.645863 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.647272 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.647416 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:46.647477 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:46.647822 (+ 345us) server_negotiation.cc:299] Negotiation successful
0925 09:26:46.647949 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":471,"threads_started":1}
I20260925 09:26:46.648351 12616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9a11271fd7449c49aa14e49933158f6" candidate_uuid: "04faf4158bb24bfb8333efe88b849851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146f2508850b4ffd81f52247de045025" is_pre_election: true
I20260925 09:26:46.648552 12616 raft_consensus.cc:2454] T d9a11271fd7449c49aa14e49933158f6 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04faf4158bb24bfb8333efe88b849851 in term 0.
I20260925 09:26:46.648795 12893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9a11271fd7449c49aa14e49933158f6" candidate_uuid: "04faf4158bb24bfb8333efe88b849851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "739baee21b8141b6a16b9b553c7c26b3" is_pre_election: true
I20260925 09:26:46.649037 12893 raft_consensus.cc:2454] T d9a11271fd7449c49aa14e49933158f6 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04faf4158bb24bfb8333efe88b849851 in term 0.
I20260925 09:26:46.649046 12683 leader_election.cc:302] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04faf4158bb24bfb8333efe88b849851, 146f2508850b4ffd81f52247de045025; no voters:
I20260925 09:26:46.649425 13058 raft_consensus.cc:2781] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:26:46.649555 13058 raft_consensus.cc:482] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:26:46.649662 13058 raft_consensus.cc:3037] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:46.651957 13058 raft_consensus.cc:504] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:46.652484 13058 leader_election.cc:288] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [CANDIDATE]: Term 1 election: Requested vote from peers 146f2508850b4ffd81f52247de045025 (127.10.114.65:35275), 739baee21b8141b6a16b9b553c7c26b3 (127.10.114.67:41333)
I20260925 09:26:46.652958 12616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9a11271fd7449c49aa14e49933158f6" candidate_uuid: "04faf4158bb24bfb8333efe88b849851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146f2508850b4ffd81f52247de045025"
I20260925 09:26:46.652958 12893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9a11271fd7449c49aa14e49933158f6" candidate_uuid: "04faf4158bb24bfb8333efe88b849851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "739baee21b8141b6a16b9b553c7c26b3"
I20260925 09:26:46.653142 12616 raft_consensus.cc:3037] T d9a11271fd7449c49aa14e49933158f6 P 146f2508850b4ffd81f52247de045025 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:46.653142 12893 raft_consensus.cc:3037] T d9a11271fd7449c49aa14e49933158f6 P 739baee21b8141b6a16b9b553c7c26b3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:46.656056 12893 raft_consensus.cc:2454] T d9a11271fd7449c49aa14e49933158f6 P 739baee21b8141b6a16b9b553c7c26b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04faf4158bb24bfb8333efe88b849851 in term 1.
I20260925 09:26:46.656056 12616 raft_consensus.cc:2454] T d9a11271fd7449c49aa14e49933158f6 P 146f2508850b4ffd81f52247de045025 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04faf4158bb24bfb8333efe88b849851 in term 1.
I20260925 09:26:46.656422 12683 leader_election.cc:302] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 04faf4158bb24bfb8333efe88b849851, 146f2508850b4ffd81f52247de045025, 739baee21b8141b6a16b9b553c7c26b3; no voters:
I20260925 09:26:46.656740 13058 raft_consensus.cc:2781] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:26:46.657174 13058 raft_consensus.cc:686] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [term 1 LEADER]: Becoming Leader. State: Replica: 04faf4158bb24bfb8333efe88b849851, State: Running, Role: LEADER
I20260925 09:26:46.657509 13058 consensus_queue.cc:237] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } }
I20260925 09:26:46.660319 12454 catalog_manager.cc:5495] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04faf4158bb24bfb8333efe88b849851 (127.10.114.66). New cstate: current_term: 1 leader_uuid: "04faf4158bb24bfb8333efe88b849851" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04faf4158bb24bfb8333efe88b849851" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 } health_report { overall_health: UNKNOWN } } }
I20260925 09:26:46.688880 13073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:46.681945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45310 (local address 127.10.114.66:39731)
0925 09:26:46.682968 (+ 1023us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:46.682974 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:46.683265 (+ 291us) server_negotiation.cc:407] Connection header received
0925 09:26:46.683538 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:46.683543 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:46.683641 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:46.683827 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:26:46.684317 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.685254 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.686698 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.687113 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.687206 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:46.687593 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:26:46.688240 (+ 647us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:26:46.688343 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:46.688418 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 09:26:46.688471 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":902,"thread_start_us":771,"threads_started":1}
I20260925 09:26:46.702899 13069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:46.697877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:53719 (local address 127.10.114.67:41333)
0925 09:26:46.698111 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:46.698120 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:46.698532 (+ 412us) server_negotiation.cc:407] Connection header received
0925 09:26:46.698622 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:46.698626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:46.698685 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:46.698779 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:26:46.699171 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.699923 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.701519 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.701878 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.701966 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:46.702343 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:26:46.702447 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:26:46.702513 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:46.702639 (+ 126us) server_negotiation.cc:299] Negotiation successful
0925 09:26:46.702692 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 09:26:46.702986 13076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:46.697826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:41333 (local address 127.10.114.66:53719)
0925 09:26:46.698398 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:46.698412 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:46.698537 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:46.698818 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:46.698821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:46.698829 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:26:46.699052 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:46.699058 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.700035 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.700039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:46.701409 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:46.701420 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.702013 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.702017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:46.702163 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:46.702213 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:26:46.702538 (+ 325us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:26:46.702546 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 09:26:46.702608 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 09:26:46.702693 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":302,"threads_started":1}
I20260925 09:26:46.710204 12893 raft_consensus.cc:1261] T d9a11271fd7449c49aa14e49933158f6 P 739baee21b8141b6a16b9b553c7c26b3 [term 1 FOLLOWER]: Refusing update from remote peer 04faf4158bb24bfb8333efe88b849851: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:26:46.710203 12616 raft_consensus.cc:1261] T d9a11271fd7449c49aa14e49933158f6 P 146f2508850b4ffd81f52247de045025 [term 1 FOLLOWER]: Refusing update from remote peer 04faf4158bb24bfb8333efe88b849851: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:26:46.710803 13058 consensus_queue.cc:1035] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "739baee21b8141b6a16b9b553c7c26b3" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 41333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:26:46.711048 13071 consensus_queue.cc:1035] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "146f2508850b4ffd81f52247de045025" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:26:46.738909 13073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:46.733583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:53799 (local address 127.10.114.66:39731)
0925 09:26:46.733742 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:46.733750 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:46.734533 (+ 783us) server_negotiation.cc:407] Connection header received
0925 09:26:46.734649 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:46.734652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:46.734704 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:46.734769 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:26:46.735147 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.736008 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.737593 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.737979 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.738064 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:46.738380 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:26:46.738478 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:26:46.738553 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:46.738665 (+ 112us) server_negotiation.cc:299] Negotiation successful
0925 09:26:46.738716 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 09:26:46.739012 13082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:46.733536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:39731 (local address 127.10.114.67:53799)
0925 09:26:46.734465 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:46.734479 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:46.734574 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:46.734796 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:46.734798 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:46.734806 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:26:46.735025 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:46.735031 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.736160 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.736164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:46.737472 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:46.737486 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:46.738120 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:46.738124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:46.738228 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:46.738258 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:26:46.738575 (+ 317us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:26:46.738582 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 09:26:46.738640 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 09:26:46.738724 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"thread_start_us":651,"threads_started":1}
W20260925 09:26:46.739957 13023 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 09:26:48.335990 12454 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:26:48.336225 12454 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:26:48.339715 12454 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8472853bdebe4d7f99bfaddcd21d4dbf: Sending DeleteTablet for 3 replicas of tablet d9a11271fd7449c49aa14e49933158f6
I20260925 09:26:48.340459 12591 tablet_service.cc:1514] Processing DeleteTablet for tablet d9a11271fd7449c49aa14e49933158f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:26:48 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:57892
I20260925 09:26:48.340534 12730 tablet_service.cc:1514] Processing DeleteTablet for tablet d9a11271fd7449c49aa14e49933158f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:26:48 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:45308
I20260925 09:26:48.340583 12868 tablet_service.cc:1514] Processing DeleteTablet for tablet d9a11271fd7449c49aa14e49933158f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:26:48 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:34976
I20260925 09:26:48.341131 13109 tablet_replica.cc:330] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851: stopping tablet replica
I20260925 09:26:48.341378 13108 tablet_replica.cc:330] T d9a11271fd7449c49aa14e49933158f6 P 146f2508850b4ffd81f52247de045025: stopping tablet replica
I20260925 09:26:48.341384 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 12525
I20260925 09:26:48.341565 13110 tablet_replica.cc:330] T d9a11271fd7449c49aa14e49933158f6 P 739baee21b8141b6a16b9b553c7c26b3: stopping tablet replica
I20260925 09:26:48.341583 13108 raft_consensus.cc:2229] T d9a11271fd7449c49aa14e49933158f6 P 146f2508850b4ffd81f52247de045025 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:26:48.341753 13110 raft_consensus.cc:2229] T d9a11271fd7449c49aa14e49933158f6 P 739baee21b8141b6a16b9b553c7c26b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:26:48.341799 13108 raft_consensus.cc:2258] T d9a11271fd7449c49aa14e49933158f6 P 146f2508850b4ffd81f52247de045025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:26:48.341998 13110 raft_consensus.cc:2258] T d9a11271fd7449c49aa14e49933158f6 P 739baee21b8141b6a16b9b553c7c26b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:26:48.342324 13109 raft_consensus.cc:2229] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:26:48.342729 13108 ts_tablet_manager.cc:1905] T d9a11271fd7449c49aa14e49933158f6 P 146f2508850b4ffd81f52247de045025: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:26:48.342828 13109 raft_consensus.cc:2258] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:26:48.343832 13110 ts_tablet_manager.cc:1905] T d9a11271fd7449c49aa14e49933158f6 P 739baee21b8141b6a16b9b553c7c26b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:26:48.344097 13109 ts_tablet_manager.cc:1905] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:26:48.349567 13108 ts_tablet_manager.cc:1918] T d9a11271fd7449c49aa14e49933158f6 P 146f2508850b4ffd81f52247de045025: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 09:26:48.349726 13108 log.cc:1196] T d9a11271fd7449c49aa14e49933158f6 P 146f2508850b4ffd81f52247de045025: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/d9a11271fd7449c49aa14e49933158f6
I20260925 09:26:48.350208 13108 ts_tablet_manager.cc:1939] T d9a11271fd7449c49aa14e49933158f6 P 146f2508850b4ffd81f52247de045025: Deleting consensus metadata
I20260925 09:26:48.350754 13109 ts_tablet_manager.cc:1918] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 09:26:48.350966 13109 log.cc:1196] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/d9a11271fd7449c49aa14e49933158f6
I20260925 09:26:48.351469 13109 ts_tablet_manager.cc:1939] T d9a11271fd7449c49aa14e49933158f6 P 04faf4158bb24bfb8333efe88b849851: Deleting consensus metadata
I20260925 09:26:48.351589 12440 catalog_manager.cc:4841] TS 146f2508850b4ffd81f52247de045025 (127.10.114.65:35275): tablet d9a11271fd7449c49aa14e49933158f6 (table test-table [id=afef04e9663644eb85fe47be77656c82]) successfully deleted
I20260925 09:26:48.351800 13110 ts_tablet_manager.cc:1918] T d9a11271fd7449c49aa14e49933158f6 P 739baee21b8141b6a16b9b553c7c26b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 09:26:48.352016 13110 log.cc:1196] T d9a11271fd7449c49aa14e49933158f6 P 739baee21b8141b6a16b9b553c7c26b3: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/d9a11271fd7449c49aa14e49933158f6
I20260925 09:26:48.352603 13110 ts_tablet_manager.cc:1939] T d9a11271fd7449c49aa14e49933158f6 P 739baee21b8141b6a16b9b553c7c26b3: Deleting consensus metadata
I20260925 09:26:48.352962 12438 catalog_manager.cc:4841] TS 04faf4158bb24bfb8333efe88b849851 (127.10.114.66:39731): tablet d9a11271fd7449c49aa14e49933158f6 (table test-table [id=afef04e9663644eb85fe47be77656c82]) successfully deleted
I20260925 09:26:48.353670 12441 catalog_manager.cc:4841] TS 739baee21b8141b6a16b9b553c7c26b3 (127.10.114.67:41333): tablet d9a11271fd7449c49aa14e49933158f6 (table test-table [id=afef04e9663644eb85fe47be77656c82]) successfully deleted
Sep 25 09:26:48 dist-test-slave-9pj8 krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328406, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:26:48.369859 13112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:48.344163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57904 (local address 127.10.114.65:35275)
0925 09:26:48.351536 (+ 7373us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:48.351543 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:48.352506 (+ 963us) server_negotiation.cc:407] Connection header received
0925 09:26:48.352686 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:48.352693 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:48.352777 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:48.352889 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:48.354419 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:48.355174 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:48.356552 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:48.356961 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:48.357068 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:48.364611 (+ 7543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:48.364624 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:48.364641 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:48.364675 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:48.367580 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:48.368246 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:48.368253 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:48.368255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:48.368376 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:48.368699 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:48.368702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:48.368704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:48.369018 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:48.369223 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:48.369458 (+ 235us) server_negotiation.cc:299] Negotiation successful
0925 09:26:48.369615 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7234,"thread_start_us":7107,"threads_started":1}
W20260925 09:26:48.370478 12656 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57904
W20260925 09:26:48.371066 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 12525: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:26:48.371233 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 12525
I20260925 09:26:48.394405 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 12665
Sep 25 09:26:48 dist-test-slave-9pj8 krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328406, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.114.66@KRBTEST.COM
I20260925 09:26:48.409883 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:48.395200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45324 (local address 127.10.114.66:39731)
0925 09:26:48.395705 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:48.395709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:48.395725 (+ 16us) server_negotiation.cc:407] Connection header received
0925 09:26:48.395820 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:48.395828 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:48.395905 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:48.395995 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:48.397217 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:48.398116 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:48.399554 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:48.400086 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:48.400196 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:48.404273 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:48.404288 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:48.404304 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:48.404353 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:48.407383 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:48.408142 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:48.408149 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:48.408152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:48.408277 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:48.408594 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:48.408597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:48.408599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:48.408935 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:48.409148 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:48.409350 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 09:26:48.409515 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":297,"threads_started":1}
W20260925 09:26:48.410528 12795 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45324
W20260925 09:26:48.411126 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 12665: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:26:48.411310 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 12665
I20260925 09:26:48.433120 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 12804
Sep 25 09:26:48 dist-test-slave-9pj8 krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328406, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.114.67@KRBTEST.COM
I20260925 09:26:48.447557 13114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:48.433827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34998 (local address 127.10.114.67:41333)
0925 09:26:48.434343 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:48.434347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:48.434369 (+ 22us) server_negotiation.cc:407] Connection header received
0925 09:26:48.434416 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:48.434419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:48.434473 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:48.434559 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:48.435866 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:48.436652 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:48.437911 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:48.438306 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:48.438411 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:48.442465 (+ 4054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:48.442477 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:48.442494 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:48.442526 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:48.445381 (+ 2855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:48.445986 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:48.445993 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:48.445995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:48.446125 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:48.446427 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:48.446430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:48.446432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:48.446740 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:48.446952 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:48.447149 (+ 197us) server_negotiation.cc:299] Negotiation successful
0925 09:26:48.447289 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":272,"threads_started":1}
W20260925 09:26:48.448263 12934 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34998
W20260925 09:26:48.448884 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 12804: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:26:48.449043 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 12804
I20260925 09:26:48.473635 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 12422
I20260925 09:26:48.474821 12494 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:26:48.592304 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 12422
2026-09-25T09:26:48Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8101 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:26:48 dist-test-slave-9pj8 krb5kdc[13118](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:26:48 dist-test-slave-9pj8 krb5kdc[13118](info): set up 2 sockets
Sep 25 09:26:48 dist-test-slave-9pj8 krb5kdc[13118](info): commencing operation
krb5kdc: starting...
W20260925 09:26:49.676189 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:26:49 dist-test-slave-9pj8 krb5kdc[13118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:26:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:26:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:26:49.858440 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:42521
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:44631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:42521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:50.160013 13131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:50.160260 13131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:50.160435 13131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:50.166939 13131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:26:50.167083 13131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:50.167182 13131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:50.167263 13131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:26:50.167346 13131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:26:50.176498 13131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:44631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:42521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:42521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:50.177659 13131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:50.178939 13131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:50.189769 13136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:50.189890 13139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:50.189744 13137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:50.191075 13131 server_base.cc:1034] running on GCE node
I20260925 09:26:50.191892 13131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:50.193138 13131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:50.194373 13131 hybrid_clock.cc:648] HybridClock initialized: now 1790328410194328 us; error 59 us; skew 500 ppm
I20260925 09:26:50.194700 13131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:50 dist-test-slave-9pj8 krb5kdc[13118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328410, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:50.197554 13131 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:26:50.198971 13131 webserver.cc:460] Webserver started at http://127.10.114.126:38365/ using document root <none> and password file <none>
I20260925 09:26:50.199642 13131 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:50.199819 13131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:50.200132 13131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:50.202844 13131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "716797341fbb46aa8088da1b9c1c984e"
format_stamp: "Formatted at 2026-09-25 09:26:50 on dist-test-slave-9pj8"
server_key: "5372295cfae9657dd4d26a81c5cc4426"
server_key_iv: "d4c8fc0c40ae550b812b6d369b83fdb5"
server_key_version: "encryptionkey@0"
I20260925 09:26:50.203583 13131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "716797341fbb46aa8088da1b9c1c984e"
format_stamp: "Formatted at 2026-09-25 09:26:50 on dist-test-slave-9pj8"
server_key: "5372295cfae9657dd4d26a81c5cc4426"
server_key_iv: "d4c8fc0c40ae550b812b6d369b83fdb5"
server_key_version: "encryptionkey@0"
I20260925 09:26:50.208302 13131 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260925 09:26:50.211337 13146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:50.212532 13131 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 09:26:50.212713 13131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "716797341fbb46aa8088da1b9c1c984e"
format_stamp: "Formatted at 2026-09-25 09:26:50 on dist-test-slave-9pj8"
server_key: "5372295cfae9657dd4d26a81c5cc4426"
server_key_iv: "d4c8fc0c40ae550b812b6d369b83fdb5"
server_key_version: "encryptionkey@0"
I20260925 09:26:50.212886 13131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:50.226254 13131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:50.229687 13131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:50.229943 13131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:50.247208 13131 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:42521
I20260925 09:26:50.247249 13197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:42521 every 8 connection(s)
I20260925 09:26:50.248538 13131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:26:50.252070 13198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:50.252908 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 13131
I20260925 09:26:50.253173 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:26:50.254060 10697 external_mini_cluster.cc:1428] Setting key 79580376d0c34f57fef840abefe66e0c
I20260925 09:26:50.266176 13198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e: Bootstrap starting.
Sep 25 09:26:50 dist-test-slave-9pj8 krb5kdc[13118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:50.269873 13198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:50.271095 13198 log.cc:824] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e: Log is configured to *not* fsync() on all Append() calls
I20260925 09:26:50.273872 13198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e: No bootstrap required, opened a new log
I20260925 09:26:50.276504 13201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:50.256154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57192 (local address 127.10.114.126:42521)
0925 09:26:50.256921 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:50.256931 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:50.256970 (+ 39us) server_negotiation.cc:407] Connection header received
0925 09:26:50.258519 (+ 1549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:50.258543 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:50.258902 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:50.259205 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:50.261066 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:50.262300 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:50.264172 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:50.264786 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:50.264923 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:50.269606 (+ 4683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:50.269613 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:50.269638 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:50.269685 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:50.272731 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:50.273414 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:50.273420 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:50.273426 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:50.273584 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:50.273961 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:50.273968 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:50.273989 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:50.274492 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:50.274806 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:50.275161 (+ 355us) server_negotiation.cc:299] Negotiation successful
0925 09:26:50.275534 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":355,"threads_started":1}
I20260925 09:26:50.281451 13198 raft_consensus.cc:348] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "716797341fbb46aa8088da1b9c1c984e" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42521 } }
I20260925 09:26:50.281818 13198 raft_consensus.cc:374] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:50.281931 13198 raft_consensus.cc:729] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 716797341fbb46aa8088da1b9c1c984e, State: Initialized, Role: FOLLOWER
I20260925 09:26:50.282480 13198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "716797341fbb46aa8088da1b9c1c984e" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42521 } }
I20260925 09:26:50.282742 13198 raft_consensus.cc:388] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:26:50.282879 13198 raft_consensus.cc:482] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:26:50.283061 13198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:50.285580 13198 raft_consensus.cc:504] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "716797341fbb46aa8088da1b9c1c984e" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42521 } }
I20260925 09:26:50.286069 13198 leader_election.cc:302] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 716797341fbb46aa8088da1b9c1c984e; no voters:
I20260925 09:26:50.286693 13198 leader_election.cc:288] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:26:50.286773 13203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:26:50.287438 13203 raft_consensus.cc:686] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e [term 1 LEADER]: Becoming Leader. State: Replica: 716797341fbb46aa8088da1b9c1c984e, State: Running, Role: LEADER
I20260925 09:26:50.288009 13203 consensus_queue.cc:237] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "716797341fbb46aa8088da1b9c1c984e" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42521 } }
I20260925 09:26:50.288861 13198 sys_catalog.cc:564] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:26:50.290647 13205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 716797341fbb46aa8088da1b9c1c984e. Latest consensus state: current_term: 1 leader_uuid: "716797341fbb46aa8088da1b9c1c984e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "716797341fbb46aa8088da1b9c1c984e" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42521 } } }
I20260925 09:26:50.291036 13205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e [sys.catalog]: This master's current role is: LEADER
I20260925 09:26:50.291558 13204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "716797341fbb46aa8088da1b9c1c984e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "716797341fbb46aa8088da1b9c1c984e" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42521 } } }
I20260925 09:26:50.291883 13204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e [sys.catalog]: This master's current role is: LEADER
I20260925 09:26:50.292922 13211 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:26:50.297989 13211 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:26:50.306310 13211 catalog_manager.cc:1347] Generated new cluster ID: 22c1f246aaec48fa9d78b38a40a527f9
I20260925 09:26:50.306427 13211 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:26:50.320439 13211 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:26:50.321602 13211 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:26:50.335040 13211 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 716797341fbb46aa8088da1b9c1c984e: Generated new TSK 0
I20260925 09:26:50.335870 13211 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
I20260925 09:26:50.385332 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42521
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:44631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:50.715456 13224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:50.715728 13224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:50.715901 13224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:50.722574 13224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:50.722662 13224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:50.722832 13224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:26:50.731972 13224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:44631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:50.733234 13224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:50.734440 13224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:50.746470 13229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:50.746454 13230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:50.746452 13232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:50.746803 13224 server_base.cc:1034] running on GCE node
I20260925 09:26:50.747547 13224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:50.748399 13224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:50.749577 13224 hybrid_clock.cc:648] HybridClock initialized: now 1790328410749549 us; error 31 us; skew 500 ppm
I20260925 09:26:50.749871 13224 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:50 dist-test-slave-9pj8 krb5kdc[13118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328410, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:50.752875 13224 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:26:50.754199 13224 webserver.cc:460] Webserver started at http://127.10.114.65:32769/ using document root <none> and password file <none>
I20260925 09:26:50.754812 13224 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:50.754933 13224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:50.755231 13224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:50.757937 13224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "98a8cb94b4fc426ea6ba835cf772aaf1"
format_stamp: "Formatted at 2026-09-25 09:26:50 on dist-test-slave-9pj8"
server_key: "e09e1820922583b1b114f088cd18a503"
server_key_iv: "e5c974e55ff7f289156214c9f815568c"
server_key_version: "encryptionkey@0"
I20260925 09:26:50.758657 13224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "98a8cb94b4fc426ea6ba835cf772aaf1"
format_stamp: "Formatted at 2026-09-25 09:26:50 on dist-test-slave-9pj8"
server_key: "e09e1820922583b1b114f088cd18a503"
server_key_iv: "e5c974e55ff7f289156214c9f815568c"
server_key_version: "encryptionkey@0"
I20260925 09:26:50.763335 13224 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260925 09:26:50.766566 13239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:50.767988 13224 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:26:50.768167 13224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "98a8cb94b4fc426ea6ba835cf772aaf1"
format_stamp: "Formatted at 2026-09-25 09:26:50 on dist-test-slave-9pj8"
server_key: "e09e1820922583b1b114f088cd18a503"
server_key_iv: "e5c974e55ff7f289156214c9f815568c"
server_key_version: "encryptionkey@0"
I20260925 09:26:50.768348 13224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:50.807092 13224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:50.811470 13224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:50.811848 13224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:50.812817 13224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:26:50.814127 13224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:26:50.814208 13224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:50.814368 13224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:26:50.814446 13224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:50.847257 13224 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:40165
I20260925 09:26:50.847339 13351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:40165 every 8 connection(s)
I20260925 09:26:50.848579 13224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:26:50.853617 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 13224
I20260925 09:26:50.853829 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:26:50.854434 10697 external_mini_cluster.cc:1428] Setting key cab4320ab80fa99b9b3edaa2e7328f29
Sep 25 09:26:50 dist-test-slave-9pj8 krb5kdc[13118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328410, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:50.869071 13354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:50.850963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:55919 (local address 127.10.114.126:42521)
0925 09:26:50.851372 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:50.851376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:50.852253 (+ 877us) server_negotiation.cc:407] Connection header received
0925 09:26:50.854092 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:50.854098 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:50.854195 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:50.854305 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:50.855915 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:50.856655 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:50.858124 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:50.858503 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:50.858643 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:50.862829 (+ 4186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:50.862834 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:50.862837 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:50.862866 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:50.865372 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:50.866408 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:50.866412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:50.866414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:50.866532 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:50.867287 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:50.867290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:50.867292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:50.867529 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:50.867644 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:50.868542 (+ 898us) server_negotiation.cc:299] Negotiation successful
0925 09:26:50.868704 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":223,"threads_started":1}
I20260925 09:26:50.870929 13355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:50.851448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42521 (local address 127.10.114.65:55919)
0925 09:26:50.852112 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:50.852145 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:50.853815 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:50.854550 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:50.854562 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:50.855079 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:50.855689 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:50.855704 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:50.856808 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:50.856813 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:50.857991 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:50.858002 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:50.858730 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:50.858739 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:50.858962 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:50.859693 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:50.859717 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:50.862588 (+ 2871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:50.865526 (+ 2938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:50.865539 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:50.865559 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:50.866255 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:50.866645 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:50.866654 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:50.866679 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:50.867119 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:50.867662 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:50.867673 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:50.868123 (+ 450us) client_negotiation.cc:769] Sending connection context
0925 09:26:50.868422 (+ 299us) client_negotiation.cc:240] Negotiation successful
0925 09:26:50.868950 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":277,"threads_started":1}
I20260925 09:26:50.872617 13352 heartbeater.cc:347] Connected to a master server at 127.10.114.126:42521
I20260925 09:26:50.872944 13352 heartbeater.cc:464] Registering TS with master...
I20260925 09:26:50.873637 13352 heartbeater.cc:511] Master 127.10.114.126:42521 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.114.66@KRBTEST.COM; defaulting to no policy
I20260925 09:26:50.875578 13163 ts_manager.cc:194] Registered new tserver with Master: 98a8cb94b4fc426ea6ba835cf772aaf1 (127.10.114.65:40165)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.66@KRBTEST.COM" created.
I20260925 09:26:50.877146 13163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:55919
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
I20260925 09:26:50.899185 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42521
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:44631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:51.201995 13358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:51.202275 13358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:51.202440 13358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:51.209667 13358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:51.209805 13358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:51.209980 13358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:26:51.219458 13358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:44631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:51.220719 13358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:51.222005 13358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:51.234652 13364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:51.234688 13363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:51.235138 13366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:51.240749 13358 server_base.cc:1034] running on GCE node
I20260925 09:26:51.241293 13358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:51.242190 13358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:51.243400 13358 hybrid_clock.cc:648] HybridClock initialized: now 1790328411243344 us; error 67 us; skew 500 ppm
I20260925 09:26:51.243698 13358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:51 dist-test-slave-9pj8 krb5kdc[13118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328411, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:51.246834 13358 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:26:51.248586 13358 webserver.cc:460] Webserver started at http://127.10.114.66:46531/ using document root <none> and password file <none>
I20260925 09:26:51.249452 13358 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:51.249570 13358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:51.249859 13358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:51.253603 13358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "5e711e0c42d9454c9acb8fc2024e06f7"
format_stamp: "Formatted at 2026-09-25 09:26:51 on dist-test-slave-9pj8"
server_key: "6b56ae7e992f8fc8a2931ff3ca5b1533"
server_key_iv: "6d70b71e6fc309dcb62c4ba12fe86d44"
server_key_version: "encryptionkey@0"
I20260925 09:26:51.254480 13358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "5e711e0c42d9454c9acb8fc2024e06f7"
format_stamp: "Formatted at 2026-09-25 09:26:51 on dist-test-slave-9pj8"
server_key: "6b56ae7e992f8fc8a2931ff3ca5b1533"
server_key_iv: "6d70b71e6fc309dcb62c4ba12fe86d44"
server_key_version: "encryptionkey@0"
I20260925 09:26:51.260486 13358 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260925 09:26:51.264562 13373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:51.265986 13358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:26:51.266135 13358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "5e711e0c42d9454c9acb8fc2024e06f7"
format_stamp: "Formatted at 2026-09-25 09:26:51 on dist-test-slave-9pj8"
server_key: "6b56ae7e992f8fc8a2931ff3ca5b1533"
server_key_iv: "6d70b71e6fc309dcb62c4ba12fe86d44"
server_key_version: "encryptionkey@0"
I20260925 09:26:51.266330 13358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:51.280797 13358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:51.284859 13358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:51.285089 13358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:51.285884 13358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:26:51.287042 13358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:26:51.287108 13358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:51.287181 13358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:26:51.287225 13358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:51.318602 13358 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:33355
I20260925 09:26:51.318639 13485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:33355 every 8 connection(s)
I20260925 09:26:51.319916 13358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:26:51.324154 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 13358
I20260925 09:26:51.324335 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:26:51.324868 10697 external_mini_cluster.cc:1428] Setting key 417c8454b305a5e288b935d9e0713f19
Sep 25 09:26:51 dist-test-slave-9pj8 krb5kdc[13118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328411, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:51.341042 13354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:51.322394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:54055 (local address 127.10.114.126:42521)
0925 09:26:51.322562 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:51.322566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:51.323700 (+ 1134us) server_negotiation.cc:407] Connection header received
0925 09:26:51.325988 (+ 2288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:51.325993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:51.326077 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:51.326199 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:51.327932 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:51.328673 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:51.331076 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:51.331538 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:51.331657 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:51.335544 (+ 3887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:51.335553 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:51.335567 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:51.335596 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:51.338155 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:51.338870 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:51.338881 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:51.338883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:51.338998 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:51.339407 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:51.339412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:51.339415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:51.339679 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:51.339851 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:51.340506 (+ 655us) server_negotiation.cc:299] Negotiation successful
0925 09:26:51.340728 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:26:51.342092 13488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:51.322903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42521 (local address 127.10.114.66:54055)
0925 09:26:51.323544 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:51.323585 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:51.325720 (+ 2135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:51.326416 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:51.326424 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:51.326971 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:51.327736 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:51.327761 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:51.328843 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:51.328851 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:51.330927 (+ 2076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:51.330944 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:51.331772 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:51.331781 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:51.332162 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:51.332888 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:51.332909 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:51.335320 (+ 2411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:51.338293 (+ 2973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:51.338303 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:51.338318 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:51.338746 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:51.339104 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:51.339110 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:51.339113 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:51.339293 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:51.339838 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:51.339845 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:51.340156 (+ 311us) client_negotiation.cc:769] Sending connection context
0925 09:26:51.340416 (+ 260us) client_negotiation.cc:240] Negotiation successful
0925 09:26:51.340875 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":250,"threads_started":1}
WARNING: no policy specified for kudu/127.10.114.67@KRBTEST.COM; defaulting to no policy
I20260925 09:26:51.343493 13486 heartbeater.cc:347] Connected to a master server at 127.10.114.126:42521
I20260925 09:26:51.343842 13486 heartbeater.cc:464] Registering TS with master...
I20260925 09:26:51.344537 13486 heartbeater.cc:511] Master 127.10.114.126:42521 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.67@KRBTEST.COM" created.
I20260925 09:26:51.346084 13163 ts_manager.cc:194] Registered new tserver with Master: 5e711e0c42d9454c9acb8fc2024e06f7 (127.10.114.66:33355)
I20260925 09:26:51.347295 13163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:54055
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
I20260925 09:26:51.367321 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42521
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:44631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:51.666328 13491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:51.666551 13491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:51.666630 13491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:51.673640 13491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:51.673743 13491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:51.673846 13491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:26:51.682817 13491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:44631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:51.684022 13491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:51.685278 13491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:51.697798 13496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:51.697842 13499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:51.698441 13497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:51.699179 13491 server_base.cc:1034] running on GCE node
I20260925 09:26:51.700073 13491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:51.701543 13491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:51.702764 13491 hybrid_clock.cc:648] HybridClock initialized: now 1790328411702713 us; error 38 us; skew 500 ppm
I20260925 09:26:51.703110 13491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:51 dist-test-slave-9pj8 krb5kdc[13118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328411, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:51.706073 13491 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:26:51.707504 13491 webserver.cc:460] Webserver started at http://127.10.114.67:33927/ using document root <none> and password file <none>
I20260925 09:26:51.708238 13491 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:51.708379 13491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:51.708680 13491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:51.711401 13491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "3ef5d5fcd1894c6eb68087b56e422c0d"
format_stamp: "Formatted at 2026-09-25 09:26:51 on dist-test-slave-9pj8"
server_key: "270a0c0faa7e2cd4b3838d8b32f178f4"
server_key_iv: "c81065546bf782c7f495960dffb56b03"
server_key_version: "encryptionkey@0"
I20260925 09:26:51.712194 13491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "3ef5d5fcd1894c6eb68087b56e422c0d"
format_stamp: "Formatted at 2026-09-25 09:26:51 on dist-test-slave-9pj8"
server_key: "270a0c0faa7e2cd4b3838d8b32f178f4"
server_key_iv: "c81065546bf782c7f495960dffb56b03"
server_key_version: "encryptionkey@0"
I20260925 09:26:51.717856 13491 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 09:26:51.721481 13506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:51.723067 13491 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 09:26:51.723248 13491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "3ef5d5fcd1894c6eb68087b56e422c0d"
format_stamp: "Formatted at 2026-09-25 09:26:51 on dist-test-slave-9pj8"
server_key: "270a0c0faa7e2cd4b3838d8b32f178f4"
server_key_iv: "c81065546bf782c7f495960dffb56b03"
server_key_version: "encryptionkey@0"
I20260925 09:26:51.723448 13491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:51.737607 13491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:51.741320 13491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:51.741595 13491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:51.742580 13491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:26:51.743762 13491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:26:51.743889 13491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:51.743968 13491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:26:51.744107 13491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:51.775401 13491 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:35107
I20260925 09:26:51.775446 13618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:35107 every 8 connection(s)
I20260925 09:26:51.776695 13491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:26:51.782732 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 13491
I20260925 09:26:51.782933 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:26:51.783486 10697 external_mini_cluster.cc:1428] Setting key 0d202625805406fe99a9a7a118db52de
Sep 25 09:26:51 dist-test-slave-9pj8 krb5kdc[13118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328411, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:51.795887 13354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:51.779198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:56245 (local address 127.10.114.126:42521)
0925 09:26:51.779395 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:51.779401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:51.780768 (+ 1367us) server_negotiation.cc:407] Connection header received
0925 09:26:51.782542 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:51.782546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:51.782611 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:51.782697 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:51.784600 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:51.785550 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:51.787025 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:51.787613 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:51.787766 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:51.791082 (+ 3316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:51.791086 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:51.791089 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:51.791117 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:51.793413 (+ 2296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:51.794085 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:51.794088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:51.794090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:51.794203 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:51.794569 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:51.794572 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:51.794574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:51.794793 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:51.794894 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:51.795461 (+ 567us) server_negotiation.cc:299] Negotiation successful
0925 09:26:51.795611 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 09:26:51.796855 13621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:51.779744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42521 (local address 127.10.114.67:56245)
0925 09:26:51.780614 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:51.780666 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:51.782251 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:51.782975 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:51.782992 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:51.783694 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:51.784433 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:51.784456 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:51.785686 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:51.785692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:51.786876 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:51.786886 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:51.787846 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:51.787855 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:51.788078 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:51.788709 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:51.788729 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:51.790907 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:51.793528 (+ 2621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:51.793534 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:51.793544 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:51.793973 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:51.794300 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:51.794303 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:51.794305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:51.794468 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:51.794894 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:51.794900 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:51.795177 (+ 277us) client_negotiation.cc:769] Sending connection context
0925 09:26:51.795369 (+ 192us) client_negotiation.cc:240] Negotiation successful
0925 09:26:51.795688 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":351,"threads_started":1}
I20260925 09:26:51.798346 13619 heartbeater.cc:347] Connected to a master server at 127.10.114.126:42521
I20260925 09:26:51.798623 13619 heartbeater.cc:464] Registering TS with master...
I20260925 09:26:51.799322 13619 heartbeater.cc:511] Master 127.10.114.126:42521 requested a full tablet report, sending...
I20260925 09:26:51.800637 13163 ts_manager.cc:194] Registered new tserver with Master: 3ef5d5fcd1894c6eb68087b56e422c0d (127.10.114.67:35107)
I20260925 09:26:51.801424 13163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:56245
I20260925 09:26:51.810377 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 09:26:51.839053 13354 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:26:51.836348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57208 (local address 127.10.114.126:42521)
0925 09:26:51.836592 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:51.836600 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:51.836995 (+ 395us) server_negotiation.cc:407] Connection header received
0925 09:26:51.837228 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:51.837239 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:51.837320 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:51.837435 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:51.838755 (+ 1320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57208: BlockingRecv error: recv got EOF from 127.0.0.1:57208 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:26:51.842005 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 13224
W20260925 09:26:51.844676 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 13224: Not authorized: Client connection negotiation failed: client connection to 127.10.114.65:40165: server requires authentication, but client does not have Kerberos credentials available
I20260925 09:26:51.844854 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 13224
I20260925 09:26:51.857373 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 13358
W20260925 09:26:51.859869 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 13358: Not authorized: Client connection negotiation failed: client connection to 127.10.114.66:33355: server requires authentication, but client does not have Kerberos credentials available
I20260925 09:26:51.860061 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 13358
I20260925 09:26:51.875335 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 13491
W20260925 09:26:51.878079 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 13491: Not authorized: Client connection negotiation failed: client connection to 127.10.114.67:35107: server requires authentication, but client does not have Kerberos credentials available
I20260925 09:26:51.878182 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 13491
I20260925 09:26:51.892944 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 13131
I20260925 09:26:51.894023 13193 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:26:51.990839 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 13131
2026-09-25T09:26:52Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3395 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:26:52 dist-test-slave-9pj8 krb5kdc[13636](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:26:52 dist-test-slave-9pj8 krb5kdc[13636](info): set up 2 sockets
Sep 25 09:26:52 dist-test-slave-9pj8 krb5kdc[13636](info): commencing operation
krb5kdc: starting...
W20260925 09:26:53.060448 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:26:53 dist-test-slave-9pj8 krb5kdc[13636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:26:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:26:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:26:53.238932 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:44351
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:34493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:44351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:53.548520 13649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:53.548831 13649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:53.548990 13649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:53.555951 13649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:26:53.556100 13649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:53.556192 13649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:53.556280 13649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:26:53.556360 13649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:26:53.565574 13649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:34493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:44351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:44351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:53.566742 13649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:53.568001 13649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:53.579139 13655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:53.579210 13657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:53.579238 13654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:53.579741 13649 server_base.cc:1034] running on GCE node
I20260925 09:26:53.580521 13649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:53.581735 13649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:53.582957 13649 hybrid_clock.cc:648] HybridClock initialized: now 1790328413582912 us; error 42 us; skew 500 ppm
I20260925 09:26:53.583325 13649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:53 dist-test-slave-9pj8 krb5kdc[13636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328413, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:53.592224 13649 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:26:53.593565 13649 webserver.cc:460] Webserver started at http://127.10.114.126:37761/ using document root <none> and password file <none>
I20260925 09:26:53.594187 13649 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:53.594302 13649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:53.594595 13649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:53.597389 13649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "cc7580fa9694425597734ccd819d67a7"
format_stamp: "Formatted at 2026-09-25 09:26:53 on dist-test-slave-9pj8"
server_key: "6edf13f820d65e8fda9190d2451103ff"
server_key_iv: "e1a30a2a96f8c1e27eea2348b53357a0"
server_key_version: "encryptionkey@0"
I20260925 09:26:53.598138 13649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "cc7580fa9694425597734ccd819d67a7"
format_stamp: "Formatted at 2026-09-25 09:26:53 on dist-test-slave-9pj8"
server_key: "6edf13f820d65e8fda9190d2451103ff"
server_key_iv: "e1a30a2a96f8c1e27eea2348b53357a0"
server_key_version: "encryptionkey@0"
I20260925 09:26:53.602991 13649 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260925 09:26:53.606173 13664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:53.607345 13649 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260925 09:26:53.607508 13649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "cc7580fa9694425597734ccd819d67a7"
format_stamp: "Formatted at 2026-09-25 09:26:53 on dist-test-slave-9pj8"
server_key: "6edf13f820d65e8fda9190d2451103ff"
server_key_iv: "e1a30a2a96f8c1e27eea2348b53357a0"
server_key_version: "encryptionkey@0"
I20260925 09:26:53.607697 13649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:53.645510 13649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:53.649761 13649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:53.650071 13649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:53.668030 13649 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:44351
I20260925 09:26:53.668071 13715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:44351 every 8 connection(s)
I20260925 09:26:53.669353 13649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:26:53.672672 13716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:53.673696 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 13649
I20260925 09:26:53.673940 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:26:53.674772 10697 external_mini_cluster.cc:1428] Setting key 44f539d20afc74a5f0bbbaf86f3b29d5
I20260925 09:26:53.685119 13716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7: Bootstrap starting.
I20260925 09:26:53.687881 13716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7: Neither blocks nor log segments found. Creating new log.
Sep 25 09:26:53 dist-test-slave-9pj8 krb5kdc[13636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:53.689066 13716 log.cc:824] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7: Log is configured to *not* fsync() on all Append() calls
I20260925 09:26:53.691763 13716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7: No bootstrap required, opened a new log
I20260925 09:26:53.697741 13719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:53.676737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54672 (local address 127.10.114.126:44351)
0925 09:26:53.677504 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:53.677516 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:53.677558 (+ 42us) server_negotiation.cc:407] Connection header received
0925 09:26:53.679293 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:53.679315 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:53.679741 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:53.680217 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:53.681783 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:53.683239 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:53.684626 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:53.685382 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:53.685581 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:53.689930 (+ 4349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:53.689943 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:53.689963 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:53.690007 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:53.693419 (+ 3412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:53.694215 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:53.694221 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:53.694227 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:53.694402 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:53.694773 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:53.694776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:53.694778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:53.695506 (+ 728us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:53.695732 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:53.696241 (+ 509us) server_negotiation.cc:299] Negotiation successful
0925 09:26:53.696686 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":352,"threads_started":1}
I20260925 09:26:53.700244 13716 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc7580fa9694425597734ccd819d67a7" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44351 } }
I20260925 09:26:53.700673 13716 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:53.700781 13716 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc7580fa9694425597734ccd819d67a7, State: Initialized, Role: FOLLOWER
I20260925 09:26:53.701365 13716 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc7580fa9694425597734ccd819d67a7" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44351 } }
I20260925 09:26:53.701591 13716 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:26:53.701736 13716 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:26:53.701789 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:26:53.701891 13716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:53.704955 13716 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc7580fa9694425597734ccd819d67a7" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44351 } }
I20260925 09:26:53.705508 13716 leader_election.cc:302] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc7580fa9694425597734ccd819d67a7; no voters:
I20260925 09:26:53.706140 13716 leader_election.cc:288] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:26:53.706302 13721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:26:53.711576 13721 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7 [term 1 LEADER]: Becoming Leader. State: Replica: cc7580fa9694425597734ccd819d67a7, State: Running, Role: LEADER
I20260925 09:26:53.712273 13721 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc7580fa9694425597734ccd819d67a7" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44351 } }
I20260925 09:26:53.712522 13716 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:26:53.715018 13723 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc7580fa9694425597734ccd819d67a7. Latest consensus state: current_term: 1 leader_uuid: "cc7580fa9694425597734ccd819d67a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc7580fa9694425597734ccd819d67a7" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44351 } } }
I20260925 09:26:53.715628 13722 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc7580fa9694425597734ccd819d67a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc7580fa9694425597734ccd819d67a7" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44351 } } }
I20260925 09:26:53.715977 13722 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7 [sys.catalog]: This master's current role is: LEADER
I20260925 09:26:53.716511 13723 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7 [sys.catalog]: This master's current role is: LEADER
I20260925 09:26:53.724578 13728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:26:53.731318 13728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:26:53.746663 13728 catalog_manager.cc:1347] Generated new cluster ID: 0e65204eb05f4d18aaec1815f5e00b5f
I20260925 09:26:53.746774 13728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:26:53.757074 13728 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:26:53.758878 13728 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:26:53.765337 13719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:53.758753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54678 (local address 127.10.114.126:44351)
0925 09:26:53.759140 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:53.759146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:53.759163 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:26:53.759218 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:53.759225 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:53.759277 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:53.759381 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:53.760406 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:53.761141 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:53.762354 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:53.762755 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:53.762885 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:53.764069 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:53.764073 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:53.764076 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:26:53.764653 (+ 577us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:53.764710 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:26:53.764733 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:53.764849 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:53.764970 (+ 121us) server_negotiation.cc:299] Negotiation successful
0925 09:26:53.765046 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265}
I20260925 09:26:53.776322 13728 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc7580fa9694425597734ccd819d67a7: Generated new TSK 0
I20260925 09:26:53.777029 13728 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:26:53.784219 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 13649
I20260925 09:26:53.785257 13711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:26:53.871534 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 13649
2026-09-25T09:26:53Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1876 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:26:53 dist-test-slave-9pj8 krb5kdc[13750](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:26:53 dist-test-slave-9pj8 krb5kdc[13750](info): set up 2 sockets
Sep 25 09:26:53 dist-test-slave-9pj8 krb5kdc[13750](info): commencing operation
krb5kdc: starting...
W20260925 09:26:54.928215 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.004s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:26:55 dist-test-slave-9pj8 krb5kdc[13750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:26:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:26:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:26:55.103109 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:44803
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:35003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:44803
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:55.409528 13763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:55.409801 13763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:55.410020 13763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:55.416653 13763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:26:55.416780 13763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:55.416888 13763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:55.417001 13763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:26:55.417093 13763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:26:55.426206 13763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:35003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:44803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:44803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:55.427424 13763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:55.428700 13763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:55.440411 13769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:55.440456 13771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:55.440478 13768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:55.441488 13763 server_base.cc:1034] running on GCE node
I20260925 09:26:55.442112 13763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:55.443389 13763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:55.444582 13763 hybrid_clock.cc:648] HybridClock initialized: now 1790328415444550 us; error 28 us; skew 500 ppm
I20260925 09:26:55.444859 13763 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:55 dist-test-slave-9pj8 krb5kdc[13750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328415, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:55.447679 13763 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:26:55.448983 13763 webserver.cc:460] Webserver started at http://127.10.114.126:34807/ using document root <none> and password file <none>
I20260925 09:26:55.449570 13763 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:55.449661 13763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:55.449982 13763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:55.452687 13763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "de39c62a456040e5b65e15b0a307d154"
format_stamp: "Formatted at 2026-09-25 09:26:55 on dist-test-slave-9pj8"
server_key: "b031564aaa4b71f42201b4501d7dc8d9"
server_key_iv: "981fdbdbf8a71a0a99935a540015c1d5"
server_key_version: "encryptionkey@0"
I20260925 09:26:55.453384 13763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "de39c62a456040e5b65e15b0a307d154"
format_stamp: "Formatted at 2026-09-25 09:26:55 on dist-test-slave-9pj8"
server_key: "b031564aaa4b71f42201b4501d7dc8d9"
server_key_iv: "981fdbdbf8a71a0a99935a540015c1d5"
server_key_version: "encryptionkey@0"
I20260925 09:26:55.458170 13763 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:26:55.461619 13778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:55.462823 13763 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 09:26:55.462991 13763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "de39c62a456040e5b65e15b0a307d154"
format_stamp: "Formatted at 2026-09-25 09:26:55 on dist-test-slave-9pj8"
server_key: "b031564aaa4b71f42201b4501d7dc8d9"
server_key_iv: "981fdbdbf8a71a0a99935a540015c1d5"
server_key_version: "encryptionkey@0"
I20260925 09:26:55.463174 13763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:55.486119 13763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:55.489823 13763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:55.490087 13763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:55.507632 13763 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:44803
I20260925 09:26:55.507668 13829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:44803 every 8 connection(s)
I20260925 09:26:55.508868 13763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:26:55.512235 13830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:55.517364 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 13763
I20260925 09:26:55.517565 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:26:55.518194 10697 external_mini_cluster.cc:1428] Setting key 9a1b7c6080615bde082b9e7a3757e2f3
I20260925 09:26:55.522400 13830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154: Bootstrap starting.
I20260925 09:26:55.525894 13830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:55.527107 13830 log.cc:824] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154: Log is configured to *not* fsync() on all Append() calls
I20260925 09:26:55.530120 13830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154: No bootstrap required, opened a new log
Sep 25 09:26:55 dist-test-slave-9pj8 krb5kdc[13750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:55.538326 13830 raft_consensus.cc:348] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de39c62a456040e5b65e15b0a307d154" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44803 } }
I20260925 09:26:55.538691 13830 raft_consensus.cc:374] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:55.538803 13830 raft_consensus.cc:729] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de39c62a456040e5b65e15b0a307d154, State: Initialized, Role: FOLLOWER
I20260925 09:26:55.539492 13830 consensus_queue.cc:260] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de39c62a456040e5b65e15b0a307d154" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44803 } }
I20260925 09:26:55.539754 13830 raft_consensus.cc:388] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:26:55.539935 13830 raft_consensus.cc:482] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:26:55.540127 13830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:55.540635 13833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:55.520282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41482 (local address 127.10.114.126:44803)
0925 09:26:55.521069 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:55.521079 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:55.521117 (+ 38us) server_negotiation.cc:407] Connection header received
0925 09:26:55.522694 (+ 1577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:55.522716 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:55.523054 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:55.523353 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:55.525059 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:55.526384 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:55.528067 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:55.528832 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:55.528951 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:55.533427 (+ 4476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:55.533434 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:55.533448 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:55.533488 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:55.536561 (+ 3073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:55.537207 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:55.537218 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:55.537223 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:55.537381 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:55.537754 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:55.537759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:55.537767 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:55.538226 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:55.538548 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:55.539025 (+ 477us) server_negotiation.cc:299] Negotiation successful
0925 09:26:55.539426 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":381,"threads_started":1}
I20260925 09:26:55.542917 13830 raft_consensus.cc:504] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de39c62a456040e5b65e15b0a307d154" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44803 } }
I20260925 09:26:55.543502 13830 leader_election.cc:302] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de39c62a456040e5b65e15b0a307d154; no voters:
I20260925 09:26:55.544100 13830 leader_election.cc:288] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:26:55.544193 13835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:26:55.544981 13835 raft_consensus.cc:686] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154 [term 1 LEADER]: Becoming Leader. State: Replica: de39c62a456040e5b65e15b0a307d154, State: Running, Role: LEADER
I20260925 09:26:55.545496 13835 consensus_queue.cc:237] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de39c62a456040e5b65e15b0a307d154" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44803 } }
I20260925 09:26:55.546132 13830 sys_catalog.cc:564] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:26:55.548439 13837 sys_catalog.cc:455] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de39c62a456040e5b65e15b0a307d154. Latest consensus state: current_term: 1 leader_uuid: "de39c62a456040e5b65e15b0a307d154" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de39c62a456040e5b65e15b0a307d154" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44803 } } }
I20260925 09:26:55.548774 13837 sys_catalog.cc:458] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154 [sys.catalog]: This master's current role is: LEADER
I20260925 09:26:55.548544 13836 sys_catalog.cc:455] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de39c62a456040e5b65e15b0a307d154" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de39c62a456040e5b65e15b0a307d154" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44803 } } }
I20260925 09:26:55.551564 13836 sys_catalog.cc:458] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154 [sys.catalog]: This master's current role is: LEADER
I20260925 09:26:55.552479 13843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:26:55.556774 13843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:26:55.564879 13843 catalog_manager.cc:1347] Generated new cluster ID: 4c14fa3839414e37a101d2e362efc660
I20260925 09:26:55.564975 13843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:26:55.573915 13843 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:26:55.575084 13843 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:26:55.581336 13843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de39c62a456040e5b65e15b0a307d154: Generated new TSK 0
I20260925 09:26:55.581965 13843 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
I20260925 09:26:55.627475 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:44803
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:35003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:55.934293 13856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:55.934551 13856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:55.934653 13856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:55.941301 13856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:55.941392 13856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:55.941493 13856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:26:55.950584 13856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:35003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:44803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:55.951697 13856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:55.953045 13856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:55.965669 13861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:55.965669 13862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:55.965711 13864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:55.966717 13856 server_base.cc:1034] running on GCE node
I20260925 09:26:55.967267 13856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:55.968146 13856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:55.969331 13856 hybrid_clock.cc:648] HybridClock initialized: now 1790328415969281 us; error 46 us; skew 500 ppm
I20260925 09:26:55.969600 13856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:55 dist-test-slave-9pj8 krb5kdc[13750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328415, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:55.972677 13856 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:26:55.974041 13856 webserver.cc:460] Webserver started at http://127.10.114.65:37601/ using document root <none> and password file <none>
I20260925 09:26:55.974674 13856 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:55.974809 13856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:55.975095 13856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:55.977901 13856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "896874d633154cd08d1b308c15cc5d81"
format_stamp: "Formatted at 2026-09-25 09:26:55 on dist-test-slave-9pj8"
server_key: "42811fcba7fc8edd65fd22f57221c941"
server_key_iv: "4d4db764f2283558a0ee0f28dff2a59e"
server_key_version: "encryptionkey@0"
I20260925 09:26:55.978612 13856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "896874d633154cd08d1b308c15cc5d81"
format_stamp: "Formatted at 2026-09-25 09:26:55 on dist-test-slave-9pj8"
server_key: "42811fcba7fc8edd65fd22f57221c941"
server_key_iv: "4d4db764f2283558a0ee0f28dff2a59e"
server_key_version: "encryptionkey@0"
I20260925 09:26:55.983426 13856 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 09:26:55.986697 13871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:55.988070 13856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:26:55.988237 13856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "896874d633154cd08d1b308c15cc5d81"
format_stamp: "Formatted at 2026-09-25 09:26:55 on dist-test-slave-9pj8"
server_key: "42811fcba7fc8edd65fd22f57221c941"
server_key_iv: "4d4db764f2283558a0ee0f28dff2a59e"
server_key_version: "encryptionkey@0"
I20260925 09:26:55.988385 13856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:56.001617 13856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:56.005275 13856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:56.005566 13856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:56.006506 13856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:26:56.007716 13856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:26:56.007885 13856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:56.007980 13856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:26:56.008019 13856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:56.040892 13856 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:37119
I20260925 09:26:56.040933 13983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:37119 every 8 connection(s)
I20260925 09:26:56.042243 13856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:26:56.043816 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 13856
I20260925 09:26:56.044063 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:26:56.044848 10697 external_mini_cluster.cc:1428] Setting key 68ab35e18dd6a4f74fd708df580be36b
Sep 25 09:26:56 dist-test-slave-9pj8 krb5kdc[13750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328415, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.114.66@KRBTEST.COM; defaulting to no policy
I20260925 09:26:56.063560 13986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:56.045145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:47367 (local address 127.10.114.126:44803)
0925 09:26:56.045558 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:56.045562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:56.046761 (+ 1199us) server_negotiation.cc:407] Connection header received
0925 09:26:56.048661 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:56.048666 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:56.048742 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:56.048863 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:56.050757 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:56.051473 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:56.053831 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:56.054225 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:56.054365 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:56.057996 (+ 3631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:56.058001 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:56.058004 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:56.058032 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:56.060628 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:56.061328 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:56.061333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:56.061336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:56.061469 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:56.061890 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:56.061899 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:56.061902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:56.062156 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:56.062311 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:56.062978 (+ 667us) server_negotiation.cc:299] Negotiation successful
0925 09:26:56.063188 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":229,"threads_started":1}
I20260925 09:26:56.064666 13987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:56.045748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:44803 (local address 127.10.114.65:47367)
0925 09:26:56.046575 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:56.046636 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:56.048403 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:56.049086 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:56.049098 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:56.049816 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:56.050605 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:56.050622 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:56.051640 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:56.051645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:56.053681 (+ 2036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:56.053691 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:56.054456 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:56.054464 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:56.054699 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:56.055353 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:56.055373 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:56.057797 (+ 2424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:56.060778 (+ 2981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:56.060786 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:56.060801 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:56.061202 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:56.061579 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:56.061584 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:56.061588 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:56.061772 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:56.062285 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:56.062292 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:56.062628 (+ 336us) client_negotiation.cc:769] Sending connection context
0925 09:26:56.062889 (+ 261us) client_negotiation.cc:240] Negotiation successful
0925 09:26:56.063367 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":312,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.66@KRBTEST.COM" created.
I20260925 09:26:56.066239 13984 heartbeater.cc:347] Connected to a master server at 127.10.114.126:44803
I20260925 09:26:56.066537 13984 heartbeater.cc:464] Registering TS with master...
I20260925 09:26:56.067373 13984 heartbeater.cc:511] Master 127.10.114.126:44803 requested a full tablet report, sending...
I20260925 09:26:56.069590 13795 ts_manager.cc:194] Registered new tserver with Master: 896874d633154cd08d1b308c15cc5d81 (127.10.114.65:37119)
I20260925 09:26:56.070986 13795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:47367
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
I20260925 09:26:56.086483 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:44803
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:35003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:56.397190 13990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:56.397493 13990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:56.397663 13990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:56.404826 13990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:56.404991 13990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:56.405159 13990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:26:56.414553 13990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:35003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:44803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:56.415757 13990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:56.417048 13990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:56.430119 13995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:56.430089 13998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:56.430250 13996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:56.431193 13990 server_base.cc:1034] running on GCE node
I20260925 09:26:56.431834 13990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:56.432747 13990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:56.434031 13990 hybrid_clock.cc:648] HybridClock initialized: now 1790328416433968 us; error 61 us; skew 500 ppm
I20260925 09:26:56.434352 13990 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:56 dist-test-slave-9pj8 krb5kdc[13750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328416, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:56.437356 13990 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:26:56.438690 13990 webserver.cc:460] Webserver started at http://127.10.114.66:36729/ using document root <none> and password file <none>
I20260925 09:26:56.439378 13990 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:56.439517 13990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:56.439870 13990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:56.442590 13990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "4102667957e3435789ae456080f2fc5e"
format_stamp: "Formatted at 2026-09-25 09:26:56 on dist-test-slave-9pj8"
server_key: "e69ddd7f1934c5d40c7a4bea24921bd9"
server_key_iv: "da2266da519e20e7de6abb784cc6ed97"
server_key_version: "encryptionkey@0"
I20260925 09:26:56.443317 13990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "4102667957e3435789ae456080f2fc5e"
format_stamp: "Formatted at 2026-09-25 09:26:56 on dist-test-slave-9pj8"
server_key: "e69ddd7f1934c5d40c7a4bea24921bd9"
server_key_iv: "da2266da519e20e7de6abb784cc6ed97"
server_key_version: "encryptionkey@0"
I20260925 09:26:56.448056 13990 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260925 09:26:56.451298 14005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:56.452641 13990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:26:56.452819 13990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "4102667957e3435789ae456080f2fc5e"
format_stamp: "Formatted at 2026-09-25 09:26:56 on dist-test-slave-9pj8"
server_key: "e69ddd7f1934c5d40c7a4bea24921bd9"
server_key_iv: "da2266da519e20e7de6abb784cc6ed97"
server_key_version: "encryptionkey@0"
I20260925 09:26:56.452996 13990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:56.477613 13990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:56.481388 13990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:56.481652 13990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:56.482517 13990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:26:56.483814 13990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:26:56.483917 13990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:56.484050 13990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:26:56.484117 13990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:56.519246 13990 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:43543
I20260925 09:26:56.519289 14117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:43543 every 8 connection(s)
I20260925 09:26:56.520889 13990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:26:56.521708 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 13990
I20260925 09:26:56.521874 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:26:56.522395 10697 external_mini_cluster.cc:1428] Setting key ccb7f755331eeffe265061c00eb831f3
Sep 25 09:26:56 dist-test-slave-9pj8 krb5kdc[13750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328416, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.114.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.67@KRBTEST.COM" created.
I20260925 09:26:56.545167 13986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:56.525055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:59357 (local address 127.10.114.126:44803)
0925 09:26:56.525218 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:56.525224 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:56.528550 (+ 3326us) server_negotiation.cc:407] Connection header received
0925 09:26:56.530392 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:56.530397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:56.530475 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:56.530593 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:56.532154 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:56.532831 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:56.535267 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:56.535648 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:56.535742 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:56.539724 (+ 3982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:56.539729 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:56.539732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:56.539761 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:56.542269 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:56.542947 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:56.542952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:56.542968 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:56.543075 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:56.543465 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:56.543469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:56.543472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:56.543765 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:56.543909 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:56.544618 (+ 709us) server_negotiation.cc:299] Negotiation successful
0925 09:26:56.544804 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 09:26:56.545984 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:56.524588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:44803 (local address 127.10.114.66:59357)
0925 09:26:56.528388 (+ 3800us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:56.528441 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:56.530141 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:56.530832 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:56.530840 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:56.531306 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:56.531985 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:56.532010 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:56.532987 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:56.532995 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:56.535121 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:56.535136 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:56.535887 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:56.535896 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:56.536189 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:56.536882 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:56.536914 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:56.539507 (+ 2593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:56.542399 (+ 2892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:56.542407 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:56.542422 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:56.542827 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:56.543178 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:56.543184 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:56.543187 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:56.543352 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:56.543923 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:56.543928 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:56.544194 (+ 266us) client_negotiation.cc:769] Sending connection context
0925 09:26:56.544416 (+ 222us) client_negotiation.cc:240] Negotiation successful
0925 09:26:56.544797 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3533,"thread_start_us":3316,"threads_started":1}
I20260925 09:26:56.547587 14118 heartbeater.cc:347] Connected to a master server at 127.10.114.126:44803
I20260925 09:26:56.547926 14118 heartbeater.cc:464] Registering TS with master...
I20260925 09:26:56.548602 14118 heartbeater.cc:511] Master 127.10.114.126:44803 requested a full tablet report, sending...
I20260925 09:26:56.549968 13795 ts_manager.cc:194] Registered new tserver with Master: 4102667957e3435789ae456080f2fc5e (127.10.114.66:43543)
I20260925 09:26:56.550776 13795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:59357
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
I20260925 09:26:56.569475 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:44803
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:35003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:56.872022 14123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:56.872315 14123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:56.872484 14123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:56.879292 14123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:56.879431 14123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:56.879601 14123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:26:56.888638 14123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:35003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:44803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:56.889870 14123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:56.891099 14123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:56.904073 14129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:56.904129 14131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:56.904193 14128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:56.905251 14123 server_base.cc:1034] running on GCE node
I20260925 09:26:56.905900 14123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:56.906757 14123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:56.907995 14123 hybrid_clock.cc:648] HybridClock initialized: now 1790328416907938 us; error 44 us; skew 500 ppm
I20260925 09:26:56.908365 14123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:56 dist-test-slave-9pj8 krb5kdc[13750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328416, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:56.911387 14123 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:26:56.912823 14123 webserver.cc:460] Webserver started at http://127.10.114.67:33971/ using document root <none> and password file <none>
I20260925 09:26:56.913625 14123 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:56.913775 14123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:56.914091 14123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:56.916834 14123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "ff80633497854eaea16b8dc5ad537ef1"
format_stamp: "Formatted at 2026-09-25 09:26:56 on dist-test-slave-9pj8"
server_key: "b357c74003f64f22315a5fa4989654fb"
server_key_iv: "1dfaf4cfc0da87a80ac8de7bd0188b1c"
server_key_version: "encryptionkey@0"
I20260925 09:26:56.917589 14123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "ff80633497854eaea16b8dc5ad537ef1"
format_stamp: "Formatted at 2026-09-25 09:26:56 on dist-test-slave-9pj8"
server_key: "b357c74003f64f22315a5fa4989654fb"
server_key_iv: "1dfaf4cfc0da87a80ac8de7bd0188b1c"
server_key_version: "encryptionkey@0"
I20260925 09:26:56.922602 14123 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 09:26:56.926048 14138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:56.927325 14123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:26:56.927485 14123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "ff80633497854eaea16b8dc5ad537ef1"
format_stamp: "Formatted at 2026-09-25 09:26:56 on dist-test-slave-9pj8"
server_key: "b357c74003f64f22315a5fa4989654fb"
server_key_iv: "1dfaf4cfc0da87a80ac8de7bd0188b1c"
server_key_version: "encryptionkey@0"
I20260925 09:26:56.927680 14123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:56.949112 14123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:56.953342 14123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:56.953629 14123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:56.954566 14123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:26:56.955860 14123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:26:56.955935 14123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:56.956017 14123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:26:56.956060 14123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:56.988809 14123 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:44791
I20260925 09:26:56.988906 14250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:44791 every 8 connection(s)
I20260925 09:26:56.990072 14123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:26:56.994715 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 14123
I20260925 09:26:56.994899 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:26:56.995476 10697 external_mini_cluster.cc:1428] Setting key 997ded6a29dc65081b70758eb2bc7ed1
Sep 25 09:26:56 dist-test-slave-9pj8 krb5kdc[13750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328416, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:57.009557 13986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:56.992655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:49435 (local address 127.10.114.126:44803)
0925 09:26:56.992895 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:56.992899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:56.993988 (+ 1089us) server_negotiation.cc:407] Connection header received
0925 09:26:56.996389 (+ 2401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:56.996394 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:56.996499 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:56.996613 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:56.998087 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:56.998867 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:57.000598 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:57.000979 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:57.001100 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:57.004167 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:57.004172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:57.004176 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:57.004204 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:57.006644 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:57.007317 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:57.007325 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:57.007332 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:57.007436 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:57.007883 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:57.007888 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:57.007891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:57.008211 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:57.008382 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:57.009002 (+ 620us) server_negotiation.cc:299] Negotiation successful
0925 09:26:57.009215 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260925 09:26:57.010512 14253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:56.993150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:44803 (local address 127.10.114.67:49435)
0925 09:26:56.993836 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:56.993871 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:56.996103 (+ 2232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:56.996840 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:56.996850 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:56.997368 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:56.997933 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:56.997949 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:56.999050 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:56.999055 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:57.000431 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:57.000442 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:57.001176 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:57.001184 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:57.001396 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:57.001950 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:57.001970 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:57.003973 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:57.006796 (+ 2823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:57.006805 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:57.006819 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:57.007188 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:57.007544 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:57.007550 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:57.007553 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:57.007720 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:57.008338 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:57.008348 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:57.008665 (+ 317us) client_negotiation.cc:769] Sending connection context
0925 09:26:57.008910 (+ 245us) client_negotiation.cc:240] Negotiation successful
0925 09:26:57.009383 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":284,"threads_started":1}
I20260925 09:26:57.011721 14251 heartbeater.cc:347] Connected to a master server at 127.10.114.126:44803
I20260925 09:26:57.012017 14251 heartbeater.cc:464] Registering TS with master...
I20260925 09:26:57.012676 14251 heartbeater.cc:511] Master 127.10.114.126:44803 requested a full tablet report, sending...
I20260925 09:26:57.014071 13795 ts_manager.cc:194] Registered new tserver with Master: ff80633497854eaea16b8dc5ad537ef1 (127.10.114.67:44791)
I20260925 09:26:57.014801 13795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:49435
I20260925 09:26:57.022414 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 09:26:57 dist-test-slave-9pj8 krb5kdc[13750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328417, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 25 09:26:57 dist-test-slave-9pj8 krb5kdc[13750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328417, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:57.067492 13986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:57.053950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41486 (local address 127.10.114.126:44803)
0925 09:26:57.054174 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:57.054184 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:57.054558 (+ 374us) server_negotiation.cc:407] Connection header received
0925 09:26:57.054717 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:57.054721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:57.054783 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:57.054864 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:57.056142 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:57.056996 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:57.058188 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:57.058552 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:57.058658 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:57.062302 (+ 3644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:57.062307 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:57.062310 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:57.062337 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:57.065119 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:57.065685 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:57.065689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:57.065691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:57.065822 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:57.066188 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:57.066191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:57.066193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:57.066558 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:57.066744 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:57.066972 (+ 228us) server_negotiation.cc:299] Negotiation successful
0925 09:26:57.067113 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260925 09:26:57.068009 13795 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41486
I20260925 09:26:57.069227 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 13856
I20260925 09:26:57.073551 13984 heartbeater.cc:503] Master 127.10.114.126:44803 was elected leader, sending a full tablet report...
Sep 25 09:26:57 dist-test-slave-9pj8 krb5kdc[13750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328417, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:26:57.084111 14262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:57.070346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57484 (local address 127.10.114.65:37119)
0925 09:26:57.070976 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:57.070983 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:57.071008 (+ 25us) server_negotiation.cc:407] Connection header received
0925 09:26:57.071059 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:57.071064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:57.071255 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:57.071375 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:57.072421 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:57.073409 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:57.074784 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:57.075441 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:57.075541 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:57.078951 (+ 3410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:57.078979 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:57.078994 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:57.079049 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:57.081746 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:57.082281 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:57.082287 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:57.082289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:57.082407 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:57.082805 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:57.082809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:57.082811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:57.083136 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:57.083371 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:57.083584 (+ 213us) server_negotiation.cc:299] Negotiation successful
0925 09:26:57.083761 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":355,"threads_started":1}
W20260925 09:26:57.085216 13979 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57484
W20260925 09:26:57.086190 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 13856: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:26:57.086374 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 13856
I20260925 09:26:57.101387 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 13990
Sep 25 09:26:57 dist-test-slave-9pj8 krb5kdc[13750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328417, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.114.66@KRBTEST.COM
I20260925 09:26:57.116756 14263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:57.102083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56040 (local address 127.10.114.66:43543)
0925 09:26:57.102588 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:57.102594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:57.102615 (+ 21us) server_negotiation.cc:407] Connection header received
0925 09:26:57.102662 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:57.102685 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:57.102871 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:57.103033 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:57.104202 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:57.105213 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:57.106438 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:57.107233 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:57.107356 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:57.111422 (+ 4066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:57.111432 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:57.111447 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:57.111491 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:57.114098 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:57.114720 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:57.114728 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:57.114731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:57.114841 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:57.115188 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:57.115193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:57.115196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:57.115511 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:57.115825 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:57.116195 (+ 370us) server_negotiation.cc:299] Negotiation successful
0925 09:26:57.116461 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":258,"threads_started":1}
W20260925 09:26:57.117916 14113 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:56040
W20260925 09:26:57.118938 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 13990: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:26:57.119100 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 13990
I20260925 09:26:57.133705 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 14123
Sep 25 09:26:57 dist-test-slave-9pj8 krb5kdc[13750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328417, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.114.67@KRBTEST.COM
I20260925 09:26:57.148991 14264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:57.134509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55958 (local address 127.10.114.67:44791)
0925 09:26:57.135128 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:57.135134 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:57.135161 (+ 27us) server_negotiation.cc:407] Connection header received
0925 09:26:57.135210 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:57.135215 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:57.135408 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:57.135569 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:57.136727 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:57.137631 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:57.138793 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:57.139341 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:57.139472 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:57.143663 (+ 4191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:57.143674 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:57.143686 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:57.143719 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:57.146340 (+ 2621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:57.146978 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:57.146985 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:57.146987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:57.147096 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:57.147433 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:57.147439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:57.147442 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:57.147757 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:57.148163 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:57.148416 (+ 253us) server_negotiation.cc:299] Negotiation successful
0925 09:26:57.148667 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":332,"threads_started":1}
W20260925 09:26:57.150199 14246 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55958
W20260925 09:26:57.151229 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 14123: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:26:57.151403 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 14123
I20260925 09:26:57.166533 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 13763
I20260925 09:26:57.167641 13825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:26:57.263896 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 13763
2026-09-25T09:26:57Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3395 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:26:57 dist-test-slave-9pj8 krb5kdc[14268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:26:57 dist-test-slave-9pj8 krb5kdc[14268](info): set up 2 sockets
Sep 25 09:26:57 dist-test-slave-9pj8 krb5kdc[14268](info): commencing operation
krb5kdc: starting...
W20260925 09:26:58.312660 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:26:58 dist-test-slave-9pj8 krb5kdc[14268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:26:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:26:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:26:58.487617 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:43011
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:36257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:43011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:58.792212 14281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:58.792474 14281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:58.792610 14281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:58.799263 14281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:26:58.799379 14281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:58.799466 14281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:58.799575 14281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:26:58.799638 14281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:26:58.809026 14281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:36257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:43011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:43011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:58.810200 14281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:58.811477 14281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:58.822764 14287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:58.822810 14289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:58.822846 14286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:58.822947 14281 server_base.cc:1034] running on GCE node
I20260925 09:26:58.824076 14281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:58.825301 14281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:58.826537 14281 hybrid_clock.cc:648] HybridClock initialized: now 1790328418826471 us; error 58 us; skew 500 ppm
I20260925 09:26:58.826867 14281 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:58 dist-test-slave-9pj8 krb5kdc[14268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328418, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:58.836161 14281 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:26:58.837716 14281 webserver.cc:460] Webserver started at http://127.10.114.126:46293/ using document root <none> and password file <none>
I20260925 09:26:58.838521 14281 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:58.838660 14281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:58.838974 14281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:58.841817 14281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "2ca9c208a530480a98e169421f891179"
format_stamp: "Formatted at 2026-09-25 09:26:58 on dist-test-slave-9pj8"
server_key: "889926ea421cad1a2ac42d96dc791254"
server_key_iv: "afd2f43641aaa3d6700dfeb1cf3541f3"
server_key_version: "encryptionkey@0"
I20260925 09:26:58.842536 14281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "2ca9c208a530480a98e169421f891179"
format_stamp: "Formatted at 2026-09-25 09:26:58 on dist-test-slave-9pj8"
server_key: "889926ea421cad1a2ac42d96dc791254"
server_key_iv: "afd2f43641aaa3d6700dfeb1cf3541f3"
server_key_version: "encryptionkey@0"
I20260925 09:26:58.847260 14281 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260925 09:26:58.850636 14296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:58.851877 14281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260925 09:26:58.852056 14281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "2ca9c208a530480a98e169421f891179"
format_stamp: "Formatted at 2026-09-25 09:26:58 on dist-test-slave-9pj8"
server_key: "889926ea421cad1a2ac42d96dc791254"
server_key_iv: "afd2f43641aaa3d6700dfeb1cf3541f3"
server_key_version: "encryptionkey@0"
I20260925 09:26:58.852226 14281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:58.871743 14281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:58.876135 14281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:58.876412 14281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:58.894220 14281 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:43011
I20260925 09:26:58.894243 14347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:43011 every 8 connection(s)
I20260925 09:26:58.895558 14281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:26:58.899120 14348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:26:58.903106 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 14281
I20260925 09:26:58.903343 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:26:58.904126 10697 external_mini_cluster.cc:1428] Setting key a2b30cc06836873000ee07bcf653387e
I20260925 09:26:58.910720 14348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179: Bootstrap starting.
I20260925 09:26:58.914237 14348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179: Neither blocks nor log segments found. Creating new log.
I20260925 09:26:58.915360 14348 log.cc:824] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179: Log is configured to *not* fsync() on all Append() calls
I20260925 09:26:58.918619 14348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179: No bootstrap required, opened a new log
Sep 25 09:26:58 dist-test-slave-9pj8 krb5kdc[14268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:58.926422 14348 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca9c208a530480a98e169421f891179" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43011 } }
I20260925 09:26:58.926792 14348 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:26:58.926905 14348 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ca9c208a530480a98e169421f891179, State: Initialized, Role: FOLLOWER
I20260925 09:26:58.927119 14351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:58.906166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37344 (local address 127.10.114.126:43011)
0925 09:26:58.906948 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:58.906967 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:58.907017 (+ 50us) server_negotiation.cc:407] Connection header received
0925 09:26:58.909207 (+ 2190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:58.909229 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:58.909600 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:58.909925 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:58.911740 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:58.913157 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:58.914873 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:58.915578 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:58.915716 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:58.920268 (+ 4552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:58.920275 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:58.920292 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:58.920339 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:58.923245 (+ 2906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:58.923976 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:58.923982 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:58.923988 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:58.924136 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:58.924526 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:58.924530 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:58.924532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:58.925009 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:58.925324 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:58.925697 (+ 373us) server_negotiation.cc:299] Negotiation successful
0925 09:26:58.926068 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":371,"threads_started":1}
I20260925 09:26:58.927548 14348 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca9c208a530480a98e169421f891179" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43011 } }
I20260925 09:26:58.927853 14348 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:26:58.927997 14348 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:26:58.928169 14348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:26:58.930539 14348 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca9c208a530480a98e169421f891179" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43011 } }
I20260925 09:26:58.930940 14348 leader_election.cc:302] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ca9c208a530480a98e169421f891179; no voters:
I20260925 09:26:58.931496 14348 leader_election.cc:288] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:26:58.931685 14353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:26:58.932374 14353 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179 [term 1 LEADER]: Becoming Leader. State: Replica: 2ca9c208a530480a98e169421f891179, State: Running, Role: LEADER
I20260925 09:26:58.932894 14353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca9c208a530480a98e169421f891179" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43011 } }
I20260925 09:26:58.933670 14348 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:26:58.935477 14355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ca9c208a530480a98e169421f891179. Latest consensus state: current_term: 1 leader_uuid: "2ca9c208a530480a98e169421f891179" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca9c208a530480a98e169421f891179" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43011 } } }
I20260925 09:26:58.936235 14354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ca9c208a530480a98e169421f891179" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca9c208a530480a98e169421f891179" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43011 } } }
I20260925 09:26:58.936645 14354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179 [sys.catalog]: This master's current role is: LEADER
I20260925 09:26:58.936645 14355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179 [sys.catalog]: This master's current role is: LEADER
I20260925 09:26:58.938161 14361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:26:58.943485 14361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:26:58.952363 14361 catalog_manager.cc:1347] Generated new cluster ID: 334e7175e431430a918ca183a924bb9c
I20260925 09:26:58.952481 14361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:26:58.976079 14361 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:26:58.977939 14361 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:26:58.988400 14361 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ca9c208a530480a98e169421f891179: Generated new TSK 0
I20260925 09:26:58.989271 14361 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
I20260925 09:26:59.047359 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43011
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:36257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:59.363577 14374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:59.363891 14374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:59.364058 14374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:59.370724 14374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:59.370860 14374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:59.371031 14374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:26:59.379761 14374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:36257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43011
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:59.380919 14374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:59.382099 14374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:59.394374 14382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:59.394459 14379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:59.394405 14380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:59.395231 14374 server_base.cc:1034] running on GCE node
I20260925 09:26:59.395830 14374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:59.396694 14374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:59.397898 14374 hybrid_clock.cc:648] HybridClock initialized: now 1790328419397865 us; error 28 us; skew 500 ppm
I20260925 09:26:59.398409 14374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:59 dist-test-slave-9pj8 krb5kdc[14268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328419, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:59.401569 14374 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:26:59.402840 14374 webserver.cc:460] Webserver started at http://127.10.114.65:43197/ using document root <none> and password file <none>
I20260925 09:26:59.403460 14374 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:59.403599 14374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:59.403970 14374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:59.406697 14374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "a8ec02d8befe439a9c32f645554f9cfc"
format_stamp: "Formatted at 2026-09-25 09:26:59 on dist-test-slave-9pj8"
server_key: "7b5a836756a014d80e804bb31ae0cff9"
server_key_iv: "a64b97e1e8b531a2a1d1e2bc5a6d273c"
server_key_version: "encryptionkey@0"
I20260925 09:26:59.407426 14374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "a8ec02d8befe439a9c32f645554f9cfc"
format_stamp: "Formatted at 2026-09-25 09:26:59 on dist-test-slave-9pj8"
server_key: "7b5a836756a014d80e804bb31ae0cff9"
server_key_iv: "a64b97e1e8b531a2a1d1e2bc5a6d273c"
server_key_version: "encryptionkey@0"
I20260925 09:26:59.412164 14374 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.003s
I20260925 09:26:59.415380 14389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:59.416651 14374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:26:59.416821 14374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "a8ec02d8befe439a9c32f645554f9cfc"
format_stamp: "Formatted at 2026-09-25 09:26:59 on dist-test-slave-9pj8"
server_key: "7b5a836756a014d80e804bb31ae0cff9"
server_key_iv: "a64b97e1e8b531a2a1d1e2bc5a6d273c"
server_key_version: "encryptionkey@0"
I20260925 09:26:59.417011 14374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:59.431419 14374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:59.435504 14374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:59.435786 14374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:59.436595 14374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:26:59.438171 14374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:26:59.438273 14374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:59.438418 14374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:26:59.438484 14374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:59.470551 14374 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:42927
I20260925 09:26:59.470624 14501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:42927 every 8 connection(s)
I20260925 09:26:59.471745 14374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:26:59.476306 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 14374
I20260925 09:26:59.476529 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:26:59.477290 10697 external_mini_cluster.cc:1428] Setting key 5170a94d7c8a3ef224aa619930cae5d3
Sep 25 09:26:59 dist-test-slave-9pj8 krb5kdc[14268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328419, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:59.493505 14504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:59.474113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:43059 (local address 127.10.114.126:43011)
0925 09:26:59.474564 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:59.474569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:59.475499 (+ 930us) server_negotiation.cc:407] Connection header received
0925 09:26:59.478056 (+ 2557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:59.478062 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:59.478142 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:59.478257 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:59.479986 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:59.480672 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:59.483061 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:59.483446 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:59.483538 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:59.486965 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:59.486971 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:59.486974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:59.487002 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:59.490230 (+ 3228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:59.491027 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:59.491035 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:59.491041 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:59.491183 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:59.491574 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:59.491581 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:59.491587 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:59.491987 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:59.492140 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:59.492814 (+ 674us) server_negotiation.cc:299] Negotiation successful
0925 09:26:59.493047 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":262,"threads_started":1}
I20260925 09:26:59.493990 14505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:59.474621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43011 (local address 127.10.114.65:43059)
0925 09:26:59.475324 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:59.475360 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:59.477747 (+ 2387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:59.478480 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:59.478488 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:59.479016 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:59.479777 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:59.479832 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:59.480827 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:59.480836 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:59.482914 (+ 2078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:59.482932 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:59.483670 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:59.483697 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:59.484064 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:59.484758 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:59.484791 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:59.486764 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:59.490422 (+ 3658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:59.490431 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:59.490445 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:59.490890 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:59.491287 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:59.491291 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:59.491293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:59.491451 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:59.492130 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:59.492137 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:59.492451 (+ 314us) client_negotiation.cc:769] Sending connection context
0925 09:26:59.492649 (+ 198us) client_negotiation.cc:240] Negotiation successful
0925 09:26:59.492985 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":271,"threads_started":1}
I20260925 09:26:59.495499 14502 heartbeater.cc:347] Connected to a master server at 127.10.114.126:43011
I20260925 09:26:59.495805 14502 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.10.114.66@KRBTEST.COM; defaulting to no policy
I20260925 09:26:59.496557 14502 heartbeater.cc:511] Master 127.10.114.126:43011 requested a full tablet report, sending...
I20260925 09:26:59.498797 14313 ts_manager.cc:194] Registered new tserver with Master: a8ec02d8befe439a9c32f645554f9cfc (127.10.114.65:42927)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.66@KRBTEST.COM" created.
I20260925 09:26:59.501022 14313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:43059
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
I20260925 09:26:59.522444 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43011
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:36257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:26:59.823573 14508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:26:59.823900 14508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:26:59.824056 14508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:26:59.830700 14508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:26:59.830829 14508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:26:59.831022 14508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:26:59.840348 14508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:36257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43011
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:26:59.841483 14508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:26:59.842679 14508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:26:59.854877 14513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:59.854925 14514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:26:59.855163 14516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:26:59.855734 14508 server_base.cc:1034] running on GCE node
I20260925 09:26:59.856403 14508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:26:59.857319 14508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:26:59.858552 14508 hybrid_clock.cc:648] HybridClock initialized: now 1790328419858516 us; error 26 us; skew 500 ppm
I20260925 09:26:59.858870 14508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:26:59 dist-test-slave-9pj8 krb5kdc[14268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328419, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:26:59.861739 14508 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:26:59.863018 14508 webserver.cc:460] Webserver started at http://127.10.114.66:36791/ using document root <none> and password file <none>
I20260925 09:26:59.863641 14508 fs_manager.cc:359] Metadata directory not provided
I20260925 09:26:59.863819 14508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:26:59.864125 14508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:26:59.866818 14508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "5c5819645de74a5587337a3b1c990a1b"
format_stamp: "Formatted at 2026-09-25 09:26:59 on dist-test-slave-9pj8"
server_key: "239ce5a38e5d572af1123f3b7ecc6d87"
server_key_iv: "542adc329783d698067f2436d11ae6ae"
server_key_version: "encryptionkey@0"
I20260925 09:26:59.867549 14508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "5c5819645de74a5587337a3b1c990a1b"
format_stamp: "Formatted at 2026-09-25 09:26:59 on dist-test-slave-9pj8"
server_key: "239ce5a38e5d572af1123f3b7ecc6d87"
server_key_iv: "542adc329783d698067f2436d11ae6ae"
server_key_version: "encryptionkey@0"
I20260925 09:26:59.872656 14508 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260925 09:26:59.875916 14523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:59.877138 14508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:26:59.877292 14508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "5c5819645de74a5587337a3b1c990a1b"
format_stamp: "Formatted at 2026-09-25 09:26:59 on dist-test-slave-9pj8"
server_key: "239ce5a38e5d572af1123f3b7ecc6d87"
server_key_iv: "542adc329783d698067f2436d11ae6ae"
server_key_version: "encryptionkey@0"
I20260925 09:26:59.877473 14508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:26:59.900332 14508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:26:59.904681 14508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:26:59.904983 14508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:26:59.905967 14508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:26:59.907287 14508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:26:59.907393 14508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:59.907537 14508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:26:59.907608 14508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:26:59.942235 14508 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:33787
I20260925 09:26:59.942281 14635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:33787 every 8 connection(s)
I20260925 09:26:59.943593 14508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:26:59.947939 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 14508
I20260925 09:26:59.948146 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:26:59.948649 10697 external_mini_cluster.cc:1428] Setting key 09b6cf89a4777d00db38151154e647ad
Sep 25 09:26:59 dist-test-slave-9pj8 krb5kdc[14268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328419, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:26:59.964860 14504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:59.946172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:37755 (local address 127.10.114.126:43011)
0925 09:26:59.946393 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 09:26:59.946399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:26:59.947513 (+ 1114us) server_negotiation.cc:407] Connection header received
0925 09:26:59.949826 (+ 2313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:26:59.949833 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:26:59.949900 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:26:59.949997 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:26:59.951923 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:59.952610 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:59.954906 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:26:59.955276 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:26:59.955359 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:59.959199 (+ 3840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:26:59.959205 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:26:59.959208 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:26:59.959239 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:26:59.961700 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:59.962580 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:59.962583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:59.962585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:59.962695 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:26:59.963194 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:26:59.963197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:26:59.963199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:26:59.963414 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:26:59.963514 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 09:26:59.964482 (+ 968us) server_negotiation.cc:299] Negotiation successful
0925 09:26:59.964639 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260925 09:26:59.966481 14638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:26:59.946674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43011 (local address 127.10.114.66:37755)
0925 09:26:59.947344 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0925 09:26:59.947388 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 09:26:59.949507 (+ 2119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:26:59.950275 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:26:59.950286 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:26:59.950951 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:26:59.951725 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:59.951755 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:59.952754 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:59.952761 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:59.954762 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:26:59.954779 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:26:59.955476 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:26:59.955484 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:26:59.955803 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:26:59.956518 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:26:59.956538 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:26:59.958944 (+ 2406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:26:59.961856 (+ 2912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:59.961867 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:59.961898 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:59.962470 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:59.962813 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:26:59.962821 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:26:59.962827 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:26:59.963092 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:26:59.963543 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:26:59.963553 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:26:59.964051 (+ 498us) client_negotiation.cc:769] Sending connection context
0925 09:26:59.964377 (+ 326us) client_negotiation.cc:240] Negotiation successful
0925 09:26:59.964949 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":267,"threads_started":1}
I20260925 09:26:59.967998 14636 heartbeater.cc:347] Connected to a master server at 127.10.114.126:43011
I20260925 09:26:59.968286 14636 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.10.114.67@KRBTEST.COM; defaulting to no policy
I20260925 09:26:59.968989 14636 heartbeater.cc:511] Master 127.10.114.126:43011 requested a full tablet report, sending...
I20260925 09:26:59.970350 14313 ts_manager.cc:194] Registered new tserver with Master: 5c5819645de74a5587337a3b1c990a1b (127.10.114.66:33787)
I20260925 09:26:59.971377 14313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:37755
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
I20260925 09:26:59.995252 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43011
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:36257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:00.303167 14641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:00.303426 14641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:00.303514 14641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:00.310045 14641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:00.310139 14641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:00.310289 14641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:27:00.319521 14641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:36257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43011
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:00.320729 14641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:00.321950 14641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:00.334102 14647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:00.334061 14649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:00.334143 14646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:00.335294 14641 server_base.cc:1034] running on GCE node
I20260925 09:27:00.335841 14641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:00.336661 14641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:00.337854 14641 hybrid_clock.cc:648] HybridClock initialized: now 1790328420337817 us; error 35 us; skew 500 ppm
I20260925 09:27:00.338158 14641 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:00 dist-test-slave-9pj8 krb5kdc[14268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328420, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:00.345696 14641 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:27:00.346971 14641 webserver.cc:460] Webserver started at http://127.10.114.67:36273/ using document root <none> and password file <none>
I20260925 09:27:00.347615 14641 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:00.347750 14641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:00.348078 14641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:00.350776 14641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "721473c0545d4c63918311ba8fb0cf18"
format_stamp: "Formatted at 2026-09-25 09:27:00 on dist-test-slave-9pj8"
server_key: "019b6984b9c9387a494a22071713b948"
server_key_iv: "2fab2c4b3c10ebf1deae80a37f1f8723"
server_key_version: "encryptionkey@0"
I20260925 09:27:00.351557 14641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "721473c0545d4c63918311ba8fb0cf18"
format_stamp: "Formatted at 2026-09-25 09:27:00 on dist-test-slave-9pj8"
server_key: "019b6984b9c9387a494a22071713b948"
server_key_iv: "2fab2c4b3c10ebf1deae80a37f1f8723"
server_key_version: "encryptionkey@0"
I20260925 09:27:00.356257 14641 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260925 09:27:00.359354 14656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:00.360596 14641 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:27:00.360755 14641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "721473c0545d4c63918311ba8fb0cf18"
format_stamp: "Formatted at 2026-09-25 09:27:00 on dist-test-slave-9pj8"
server_key: "019b6984b9c9387a494a22071713b948"
server_key_iv: "2fab2c4b3c10ebf1deae80a37f1f8723"
server_key_version: "encryptionkey@0"
I20260925 09:27:00.360954 14641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:00.378288 14641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:00.382200 14641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:00.382468 14641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:00.383332 14641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:00.384651 14641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:00.384750 14641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:00.384894 14641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:00.384971 14641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:00.417613 14641 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:34767
I20260925 09:27:00.417647 14768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:34767 every 8 connection(s)
I20260925 09:27:00.418795 14641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:27:00.422333 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 14641
I20260925 09:27:00.422546 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:27:00.423177 10697 external_mini_cluster.cc:1428] Setting key 2bb143ae93e312506360082d3d399362
Sep 25 09:27:00 dist-test-slave-9pj8 krb5kdc[14268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328420, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:27:00.438171 14504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:00.421354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:33387 (local address 127.10.114.126:43011)
0925 09:27:00.421556 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:00.421560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:00.422815 (+ 1255us) server_negotiation.cc:407] Connection header received
0925 09:27:00.424836 (+ 2021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:00.424843 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:00.424946 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:00.425057 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:00.426583 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:00.427295 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:00.428815 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:00.429260 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:00.429374 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:00.432484 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:00.432489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:00.432492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:00.432521 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:00.435214 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:00.435855 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:00.435859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:00.435861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:00.436004 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:00.436383 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:00.436386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:00.436388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:00.436665 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:00.436808 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:00.437640 (+ 832us) server_negotiation.cc:299] Negotiation successful
0925 09:27:00.437865 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260925 09:27:00.439265 14771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:00.421845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43011 (local address 127.10.114.67:33387)
0925 09:27:00.422648 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:00.422691 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:00.424558 (+ 1867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:00.425302 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:00.425310 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:00.425792 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:00.426443 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:00.426461 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:00.427421 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:00.427426 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:00.428665 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:00.428676 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:00.429449 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:00.429454 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:00.429638 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:00.430230 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:00.430254 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:00.432302 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:00.435326 (+ 3024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:00.435333 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:00.435344 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:00.435717 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:00.436091 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:00.436095 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:00.436097 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:00.436291 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:00.436805 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:00.436812 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:00.437229 (+ 417us) client_negotiation.cc:769] Sending connection context
0925 09:27:00.437549 (+ 320us) client_negotiation.cc:240] Negotiation successful
0925 09:27:00.438055 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":341,"threads_started":1}
I20260925 09:27:00.440575 14769 heartbeater.cc:347] Connected to a master server at 127.10.114.126:43011
I20260925 09:27:00.440830 14769 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:00.441461 14769 heartbeater.cc:511] Master 127.10.114.126:43011 requested a full tablet report, sending...
I20260925 09:27:00.442812 14313 ts_manager.cc:194] Registered new tserver with Master: 721473c0545d4c63918311ba8fb0cf18 (127.10.114.67:34767)
I20260925 09:27:00.443524 14313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:33387
I20260925 09:27:00.450389 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 09:27:00 dist-test-slave-9pj8 krb5kdc[14268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260925 09:27:00.503875 14502 heartbeater.cc:503] Master 127.10.114.126:43011 was elected leader, sending a full tablet report...
Sep 25 09:27:00 dist-test-slave-9pj8 krb5kdc[14268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:27:00.519408 14776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:00.492824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54206 (local address 127.10.114.65:42927)
0925 09:27:00.502932 (+ 10108us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:00.502944 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:00.502981 (+ 37us) server_negotiation.cc:407] Connection header received
0925 09:27:00.503068 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:00.503081 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:00.503412 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:00.504482 (+ 1070us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:00.506411 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:00.507776 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:00.509208 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:00.509815 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:00.509957 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:00.513914 (+ 3957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:00.513924 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:00.513937 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:00.513969 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:00.516774 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:00.517394 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:00.517407 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:00.517409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:00.517526 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:00.517895 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:00.517900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:00.517903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:00.518215 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:00.518521 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:00.518800 (+ 279us) server_negotiation.cc:299] Negotiation successful
0925 09:27:00.519055 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9900,"thread_start_us":9722,"threads_started":1}
Sep 25 09:27:00 dist-test-slave-9pj8 krb5kdc[14268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:27:00.536413 14504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:00.523406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37358 (local address 127.10.114.126:43011)
0925 09:27:00.523634 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:00.523640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:00.523854 (+ 214us) server_negotiation.cc:407] Connection header received
0925 09:27:00.524046 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:00.524052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:00.524115 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:00.524218 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:00.525351 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:00.526047 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:00.527281 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:00.527700 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:00.527876 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:00.531710 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:00.531715 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:00.531718 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:00.531748 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:00.534139 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:00.534772 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:00.534776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:00.534778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:00.534896 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:00.535230 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:00.535233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:00.535235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:00.535515 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:00.535687 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:00.536059 (+ 372us) server_negotiation.cc:299] Negotiation successful
0925 09:27:00.536204 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260925 09:27:00.537020 14313 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37358
I20260925 09:27:00.538004 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 14374
I20260925 09:27:00.557885 14776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:00.538963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54218 (local address 127.10.114.65:42927)
0925 09:27:00.539180 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:00.539189 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:00.547524 (+ 8335us) server_negotiation.cc:407] Connection header received
0925 09:27:00.547652 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:00.547656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:00.547715 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:00.547842 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:00.548915 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:00.549583 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:00.550719 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:00.551138 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:00.551231 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:00.553266 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:00.553270 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:00.553273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:00.553300 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:00.555731 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:00.556299 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:00.556302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:00.556304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:00.556408 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:00.556725 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:00.556728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:00.556730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:00.557011 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:00.557170 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:00.557437 (+ 267us) server_negotiation.cc:299] Negotiation successful
0925 09:27:00.557578 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 09:27:00.558485 14497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:00.643852 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 14374
I20260925 09:27:00.659898 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 14508
Sep 25 09:27:00 dist-test-slave-9pj8 krb5kdc[14268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.66@KRBTEST.COM
I20260925 09:27:00.675853 14781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:00.660755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50614 (local address 127.10.114.66:33787)
0925 09:27:00.661305 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:00.661312 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:00.661338 (+ 26us) server_negotiation.cc:407] Connection header received
0925 09:27:00.661403 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:00.661409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:00.661652 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:00.661784 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:00.662892 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:00.664137 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:00.665501 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:00.666058 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:00.666166 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:00.670034 (+ 3868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:00.670044 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:00.670059 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:00.670089 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:00.673010 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:00.673684 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:00.673710 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:00.673712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:00.673852 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:00.674241 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:00.674262 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:00.674265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:00.674610 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:00.674907 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:00.675208 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 09:27:00.675467 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":287,"threads_started":1}
I20260925 09:27:00.677073 14631 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:00.757246 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 14508
I20260925 09:27:00.773670 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 14641
Sep 25 09:27:00 dist-test-slave-9pj8 krb5kdc[14268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.67@KRBTEST.COM
I20260925 09:27:00.789515 14783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:00.775004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47712 (local address 127.10.114.67:34767)
0925 09:27:00.775605 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:00.775613 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:00.775641 (+ 28us) server_negotiation.cc:407] Connection header received
0925 09:27:00.775700 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:00.775706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:00.775967 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:00.776137 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:00.777339 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:00.778312 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:00.779610 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:00.780214 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:00.780348 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:00.784337 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:00.784347 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:00.784360 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:00.784390 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:00.786778 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:00.787413 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:00.787425 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:00.787428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:00.787547 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:00.788013 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:00.788022 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:00.788025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:00.788338 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:00.788645 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:00.788915 (+ 270us) server_negotiation.cc:299] Negotiation successful
0925 09:27:00.789165 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":302,"threads_started":1}
I20260925 09:27:00.790767 14764 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:00.871871 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 14641
I20260925 09:27:00.887645 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 14281
I20260925 09:27:00.888794 14343 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:00.987012 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 14281
2026-09-25T09:27:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3725 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:27:01 dist-test-slave-9pj8 krb5kdc[14788](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:27:01 dist-test-slave-9pj8 krb5kdc[14788](info): set up 2 sockets
Sep 25 09:27:01 dist-test-slave-9pj8 krb5kdc[14788](info): commencing operation
krb5kdc: starting...
W20260925 09:27:02.072602 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:27:02 dist-test-slave-9pj8 krb5kdc[14788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:27:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:27:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:27:02.255539 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:39829
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:39829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:02.564884 14801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:02.565146 14801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:02.565325 14801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:02.572405 14801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:27:02.572535 14801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:02.572628 14801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:02.572715 14801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:27:02.572796 14801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:27:02.582435 14801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:39829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:39829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:02.583642 14801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:02.584941 14801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:02.596455 14807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:02.596410 14806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:02.596558 14809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:02.597378 14801 server_base.cc:1034] running on GCE node
I20260925 09:27:02.598125 14801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:02.599499 14801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:02.600754 14801 hybrid_clock.cc:648] HybridClock initialized: now 1790328422600687 us; error 63 us; skew 500 ppm
I20260925 09:27:02.601114 14801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:02 dist-test-slave-9pj8 krb5kdc[14788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328422, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:02.604290 14801 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:27:02.605432 14801 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:02.605523 14801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:02.605849 14801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:02.608726 14801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "cfef81902a7f47e49cba9d299b83ed64"
format_stamp: "Formatted at 2026-09-25 09:27:02 on dist-test-slave-9pj8"
server_key: "5ff6680aee29fdf7a7e9fa374d8d4edb"
server_key_iv: "5a6ff271a8ad40652b827d79007c8e24"
server_key_version: "encryptionkey@0"
I20260925 09:27:02.609483 14801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "cfef81902a7f47e49cba9d299b83ed64"
format_stamp: "Formatted at 2026-09-25 09:27:02 on dist-test-slave-9pj8"
server_key: "5ff6680aee29fdf7a7e9fa374d8d4edb"
server_key_iv: "5a6ff271a8ad40652b827d79007c8e24"
server_key_version: "encryptionkey@0"
I20260925 09:27:02.614326 14801 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260925 09:27:02.617717 14815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:02.619123 14801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:27:02.619339 14801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "cfef81902a7f47e49cba9d299b83ed64"
format_stamp: "Formatted at 2026-09-25 09:27:02 on dist-test-slave-9pj8"
server_key: "5ff6680aee29fdf7a7e9fa374d8d4edb"
server_key_iv: "5a6ff271a8ad40652b827d79007c8e24"
server_key_version: "encryptionkey@0"
I20260925 09:27:02.619582 14801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:02.637344 14801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:02.641731 14801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:02.642021 14801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:02.660872 14801 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:39829
I20260925 09:27:02.660892 14866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:39829 every 8 connection(s)
I20260925 09:27:02.661795 14801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:27:02.665225 14867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:02.671066 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 14801
I20260925 09:27:02.671321 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:27:02.672102 10697 external_mini_cluster.cc:1428] Setting key 75dc4220c403d7dd8dc3d01d67a764f1
I20260925 09:27:02.675082 14867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64: Bootstrap starting.
I20260925 09:27:02.678120 14867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:02.679395 14867 log.cc:824] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:02.682417 14867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64: No bootstrap required, opened a new log
Sep 25 09:27:02 dist-test-slave-9pj8 krb5kdc[14788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:27:02.690984 14867 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfef81902a7f47e49cba9d299b83ed64" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39829 } }
I20260925 09:27:02.691356 14867 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:02.691484 14867 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfef81902a7f47e49cba9d299b83ed64, State: Initialized, Role: FOLLOWER
I20260925 09:27:02.692057 14867 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfef81902a7f47e49cba9d299b83ed64" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39829 } }
I20260925 09:27:02.692293 14867 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:27:02.692435 14867 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:27:02.692615 14867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:02.694697 14867 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfef81902a7f47e49cba9d299b83ed64" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39829 } }
I20260925 09:27:02.695181 14867 leader_election.cc:302] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfef81902a7f47e49cba9d299b83ed64; no voters:
I20260925 09:27:02.695825 14867 leader_election.cc:288] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:27:02.695950 14872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:02.697507 14870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:02.674259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60758 (local address 127.10.114.126:39829)
0925 09:27:02.675256 (+ 997us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:02.675270 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:02.675315 (+ 45us) server_negotiation.cc:407] Connection header received
0925 09:27:02.677224 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:02.677245 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:02.677625 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:02.677960 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:02.679828 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:02.681603 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:02.683365 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:02.684193 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:02.684352 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:02.688757 (+ 4405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:02.688764 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:02.688777 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:02.688808 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:02.692474 (+ 3666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:02.693253 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:02.693267 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:02.693287 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:02.693484 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:02.693970 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:02.693977 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:02.693980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:02.694577 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:02.694960 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:02.695429 (+ 469us) server_negotiation.cc:299] Negotiation successful
0925 09:27:02.695982 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":456,"threads_started":1}
I20260925 09:27:02.698311 14867 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:27:02.700085 14872 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64 [term 1 LEADER]: Becoming Leader. State: Replica: cfef81902a7f47e49cba9d299b83ed64, State: Running, Role: LEADER
I20260925 09:27:02.700860 14872 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfef81902a7f47e49cba9d299b83ed64" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39829 } }
I20260925 09:27:02.704164 14874 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfef81902a7f47e49cba9d299b83ed64. Latest consensus state: current_term: 1 leader_uuid: "cfef81902a7f47e49cba9d299b83ed64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfef81902a7f47e49cba9d299b83ed64" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39829 } } }
I20260925 09:27:02.704136 14873 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfef81902a7f47e49cba9d299b83ed64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfef81902a7f47e49cba9d299b83ed64" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39829 } } }
I20260925 09:27:02.704692 14874 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:02.704713 14873 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:02.709146 14881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:27:02.713508 14881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:27:02.722267 14881 catalog_manager.cc:1347] Generated new cluster ID: 51570abe5886438d82032944fc26a244
I20260925 09:27:02.722378 14881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:27:02.737308 14881 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:27:02.738945 14881 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:27:02.752895 14881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64: Generated new TSK 0
I20260925 09:27:02.753844 14881 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
I20260925 09:27:02.811353 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39829
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:03.120393 14893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:03.120687 14893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:03.120886 14893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:03.127920 14893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:03.128052 14893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:03.128212 14893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:27:03.137239 14893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:03.138368 14893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:03.139595 14893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:03.151748 14901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:03.151831 14898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:03.151836 14899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:03.152669 14893 server_base.cc:1034] running on GCE node
I20260925 09:27:03.153221 14893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:03.154009 14893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:03.155248 14893 hybrid_clock.cc:648] HybridClock initialized: now 1790328423155211 us; error 47 us; skew 500 ppm
I20260925 09:27:03.155536 14893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:03 dist-test-slave-9pj8 krb5kdc[14788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328423, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:03.158414 14893 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:27:03.159503 14893 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:03.159632 14893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:03.159978 14893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:03.162578 14893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "b0efc5d4d392485e803f05ed0678a418"
format_stamp: "Formatted at 2026-09-25 09:27:03 on dist-test-slave-9pj8"
server_key: "0d0092013693b72069ca5f89b7188f92"
server_key_iv: "efd7be86d19cd6abc62d1eb79977545e"
server_key_version: "encryptionkey@0"
I20260925 09:27:03.163316 14893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "b0efc5d4d392485e803f05ed0678a418"
format_stamp: "Formatted at 2026-09-25 09:27:03 on dist-test-slave-9pj8"
server_key: "0d0092013693b72069ca5f89b7188f92"
server_key_iv: "efd7be86d19cd6abc62d1eb79977545e"
server_key_version: "encryptionkey@0"
I20260925 09:27:03.167666 14893 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260925 09:27:03.170620 14907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:03.171758 14893 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 09:27:03.171993 14893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "b0efc5d4d392485e803f05ed0678a418"
format_stamp: "Formatted at 2026-09-25 09:27:03 on dist-test-slave-9pj8"
server_key: "0d0092013693b72069ca5f89b7188f92"
server_key_iv: "efd7be86d19cd6abc62d1eb79977545e"
server_key_version: "encryptionkey@0"
I20260925 09:27:03.172197 14893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:03.182034 14893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:03.185487 14893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:03.185738 14893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:03.186594 14893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:03.187760 14893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:03.187922 14893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:03.188047 14893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:03.188129 14893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:03.220211 14893 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:42819
I20260925 09:27:03.220294 15019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:42819 every 8 connection(s)
I20260925 09:27:03.221087 14893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:27:03.228752 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 14893
I20260925 09:27:03.228981 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:27:03.229557 10697 external_mini_cluster.cc:1428] Setting key 272ab82b1cb99d0a43e075a39d32a5b8
Sep 25 09:27:03 dist-test-slave-9pj8 krb5kdc[14788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328423, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:27:03.241557 15022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:03.223625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:51569 (local address 127.10.114.126:39829)
0925 09:27:03.224049 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:03.224053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:03.225101 (+ 1048us) server_negotiation.cc:407] Connection header received
0925 09:27:03.226801 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:03.226806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:03.226890 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:03.227013 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:03.228553 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:03.229515 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:03.231407 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:03.231826 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:03.231922 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:03.236094 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:03.236099 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:03.236102 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:03.236131 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:03.238608 (+ 2477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:03.239323 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:03.239329 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:03.239332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:03.239448 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:03.239878 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:03.239883 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:03.239886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:03.240153 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:03.240306 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:03.240942 (+ 636us) server_negotiation.cc:299] Negotiation successful
0925 09:27:03.241165 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":230,"threads_started":1}
I20260925 09:27:03.242480 15023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:03.224257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39829 (local address 127.10.114.65:51569)
0925 09:27:03.224954 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:03.224991 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:03.226553 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:03.227228 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:03.227236 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:03.227733 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:03.228373 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:03.228400 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:03.229672 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:03.229678 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:03.231258 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:03.231270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:03.232047 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:03.232058 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:03.232391 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:03.233190 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:03.233223 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:03.235875 (+ 2652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:03.238757 (+ 2882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:03.238765 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:03.238780 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:03.239164 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:03.239557 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:03.239563 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:03.239566 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:03.239743 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:03.240284 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:03.240291 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:03.240617 (+ 326us) client_negotiation.cc:769] Sending connection context
0925 09:27:03.240858 (+ 241us) client_negotiation.cc:240] Negotiation successful
0925 09:27:03.241364 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":265,"threads_started":1}
I20260925 09:27:03.243973 15020 heartbeater.cc:347] Connected to a master server at 127.10.114.126:39829
I20260925 09:27:03.244300 15020 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:03.244891 15020 heartbeater.cc:511] Master 127.10.114.126:39829 requested a full tablet report, sending...
I20260925 09:27:03.246815 14832 ts_manager.cc:194] Registered new tserver with Master: b0efc5d4d392485e803f05ed0678a418 (127.10.114.65:42819)
WARNING: no policy specified for kudu/127.10.114.66@KRBTEST.COM; defaulting to no policy
I20260925 09:27:03.248294 14832 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:51569
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
I20260925 09:27:03.271183 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39829
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:03.577414 15026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:03.577690 15026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:03.577780 15026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:03.584692 15026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:03.584839 15026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:03.584964 15026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:27:03.594352 15026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:03.595539 15026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:03.596796 15026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:03.609901 15034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:03.610551 15032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:03.610602 15031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:03.611043 15026 server_base.cc:1034] running on GCE node
I20260925 09:27:03.611552 15026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:03.612443 15026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:03.613662 15026 hybrid_clock.cc:648] HybridClock initialized: now 1790328423613608 us; error 27 us; skew 500 ppm
I20260925 09:27:03.613976 15026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:03 dist-test-slave-9pj8 krb5kdc[14788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328423, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:03.617162 15026 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:27:03.618312 15026 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:03.618396 15026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:03.618709 15026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:03.622457 15026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "3fc64ce9bc40435b8ded2190e1bb3af7"
format_stamp: "Formatted at 2026-09-25 09:27:03 on dist-test-slave-9pj8"
server_key: "37fdb9d39fcdaf5fd3bbacf2ff7775ea"
server_key_iv: "d418513dfaa11b2dc0f0280975839296"
server_key_version: "encryptionkey@0"
I20260925 09:27:03.623319 15026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "3fc64ce9bc40435b8ded2190e1bb3af7"
format_stamp: "Formatted at 2026-09-25 09:27:03 on dist-test-slave-9pj8"
server_key: "37fdb9d39fcdaf5fd3bbacf2ff7775ea"
server_key_iv: "d418513dfaa11b2dc0f0280975839296"
server_key_version: "encryptionkey@0"
I20260925 09:27:03.628350 15026 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 09:27:03.631554 15040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:03.632915 15026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:27:03.633095 15026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "3fc64ce9bc40435b8ded2190e1bb3af7"
format_stamp: "Formatted at 2026-09-25 09:27:03 on dist-test-slave-9pj8"
server_key: "37fdb9d39fcdaf5fd3bbacf2ff7775ea"
server_key_iv: "d418513dfaa11b2dc0f0280975839296"
server_key_version: "encryptionkey@0"
I20260925 09:27:03.633337 15026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:03.647047 15026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:03.650832 15026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:03.651098 15026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:03.652015 15026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:03.653225 15026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:03.653339 15026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:03.653456 15026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:03.653545 15026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:03.687239 15026 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:39291
I20260925 09:27:03.687323 15152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:39291 every 8 connection(s)
I20260925 09:27:03.688196 15026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:27:03.696623 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 15026
I20260925 09:27:03.696851 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:27:03.697317 10697 external_mini_cluster.cc:1428] Setting key 1dd793f9b5e78575f99186d8d55d5fc0
Sep 25 09:27:03 dist-test-slave-9pj8 krb5kdc[14788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328423, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:27:03.710114 15022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:03.690742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:59531 (local address 127.10.114.126:39829)
0925 09:27:03.690939 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:03.690943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:03.692192 (+ 1249us) server_negotiation.cc:407] Connection header received
0925 09:27:03.694083 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:03.694089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:03.694190 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:03.694311 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:03.695848 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:03.696622 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:03.698680 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:03.699176 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:03.699354 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:03.702760 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:03.702771 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:03.702775 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:03.702804 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:03.706345 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:03.707390 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:03.707399 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:03.707406 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:03.707610 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:03.708062 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:03.708070 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:03.708077 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:03.708535 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:03.708694 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:03.709352 (+ 658us) server_negotiation.cc:299] Negotiation successful
0925 09:27:03.709649 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:27:03.710623 15155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:03.691256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39829 (local address 127.10.114.66:59531)
0925 09:27:03.692018 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:03.692057 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:03.693798 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:03.694554 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:03.694562 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:03.695079 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:03.695649 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:03.695667 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:03.696797 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:03.696803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:03.698530 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:03.698545 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:03.699433 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:03.699441 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:03.699816 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:03.700508 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:03.700541 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:03.702538 (+ 1997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:03.706571 (+ 4033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:03.706582 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:03.706617 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:03.707256 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:03.707725 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:03.707728 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:03.707731 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:03.707943 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:03.708669 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:03.708675 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:03.708999 (+ 324us) client_negotiation.cc:769] Sending connection context
0925 09:27:03.709204 (+ 205us) client_negotiation.cc:240] Negotiation successful
0925 09:27:03.709605 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":288,"threads_started":1}
I20260925 09:27:03.712067 15153 heartbeater.cc:347] Connected to a master server at 127.10.114.126:39829
I20260925 09:27:03.712356 15153 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:03.712982 15153 heartbeater.cc:511] Master 127.10.114.126:39829 requested a full tablet report, sending...
I20260925 09:27:03.714368 14832 ts_manager.cc:194] Registered new tserver with Master: 3fc64ce9bc40435b8ded2190e1bb3af7 (127.10.114.66:39291)
I20260925 09:27:03.715176 14832 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:59531
WARNING: no policy specified for kudu/127.10.114.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
I20260925 09:27:03.742745 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39829
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:04.058027 15158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:04.058274 15158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:04.058364 15158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:04.065222 15158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:04.065340 15158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:04.065578 15158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:27:04.075567 15158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:04.076838 15158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:04.078074 15158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:04.090432 15164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:04.090404 15166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:04.090698 15163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:04.091162 15158 server_base.cc:1034] running on GCE node
I20260925 09:27:04.091715 15158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:04.092600 15158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:04.093828 15158 hybrid_clock.cc:648] HybridClock initialized: now 1790328424093791 us; error 52 us; skew 500 ppm
I20260925 09:27:04.094172 15158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:04 dist-test-slave-9pj8 krb5kdc[14788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328424, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:04.100570 15158 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:27:04.101616 15158 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:04.101698 15158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:04.101928 15158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:04.104646 15158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "324d286fb92744258cc1522647c347bb"
format_stamp: "Formatted at 2026-09-25 09:27:04 on dist-test-slave-9pj8"
server_key: "828be97059040238bfabe2a4c286852e"
server_key_iv: "f0d35b0c5173e95039e0afc4a4beae72"
server_key_version: "encryptionkey@0"
I20260925 09:27:04.105358 15158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "324d286fb92744258cc1522647c347bb"
format_stamp: "Formatted at 2026-09-25 09:27:04 on dist-test-slave-9pj8"
server_key: "828be97059040238bfabe2a4c286852e"
server_key_iv: "f0d35b0c5173e95039e0afc4a4beae72"
server_key_version: "encryptionkey@0"
I20260925 09:27:04.109946 15158 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:27:04.113101 15172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:04.114282 15158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260925 09:27:04.114398 15158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "324d286fb92744258cc1522647c347bb"
format_stamp: "Formatted at 2026-09-25 09:27:04 on dist-test-slave-9pj8"
server_key: "828be97059040238bfabe2a4c286852e"
server_key_iv: "f0d35b0c5173e95039e0afc4a4beae72"
server_key_version: "encryptionkey@0"
I20260925 09:27:04.114562 15158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:04.136159 15158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:04.140272 15158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:04.140478 15158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:04.141292 15158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:04.142482 15158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:04.142575 15158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:04.142649 15158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:04.142701 15158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:04.174453 15158 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:34363
I20260925 09:27:04.174538 15284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:34363 every 8 connection(s)
I20260925 09:27:04.175235 15158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:27:04.180534 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 15158
I20260925 09:27:04.180723 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:27:04.181247 10697 external_mini_cluster.cc:1428] Setting key a8a1c35a732e28129581c88ee8acaf04
Sep 25 09:27:04 dist-test-slave-9pj8 krb5kdc[14788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328424, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:27:04.195873 15022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.177765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:42373 (local address 127.10.114.126:39829)
0925 09:27:04.177962 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:04.177968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:04.179162 (+ 1194us) server_negotiation.cc:407] Connection header received
0925 09:27:04.181329 (+ 2167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:04.181338 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:04.181422 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:04.181507 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:04.183899 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.184723 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.186220 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.186603 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.186739 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.190411 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:04.190416 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:04.190419 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:04.190449 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:04.192908 (+ 2459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:04.193611 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:04.193615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:04.193617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:04.193743 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:04.194144 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:04.194151 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:04.194155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:04.194519 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:04.194677 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:04.195308 (+ 631us) server_negotiation.cc:299] Negotiation successful
0925 09:27:04.195523 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 09:27:04.196904 15287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.178317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39829 (local address 127.10.114.67:42373)
0925 09:27:04.179004 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:04.179046 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:04.181031 (+ 1985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:04.181773 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:04.181783 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:04.182381 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:04.183104 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:04.183123 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.184863 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.184868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:04.186072 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:04.186083 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.186812 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.186822 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:04.187052 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.187881 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:04.187903 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:04.190229 (+ 2326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:04.193026 (+ 2797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:04.193032 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:04.193049 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:04.193437 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:04.193852 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:04.193858 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:04.193861 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:04.194017 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:04.194647 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:04.194655 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:04.194949 (+ 294us) client_negotiation.cc:769] Sending connection context
0925 09:27:04.195228 (+ 279us) client_negotiation.cc:240] Negotiation successful
0925 09:27:04.195704 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":273,"threads_started":1}
I20260925 09:27:04.198225 15285 heartbeater.cc:347] Connected to a master server at 127.10.114.126:39829
I20260925 09:27:04.198513 15285 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:04.199124 15285 heartbeater.cc:511] Master 127.10.114.126:39829 requested a full tablet report, sending...
I20260925 09:27:04.200448 14832 ts_manager.cc:194] Registered new tserver with Master: 324d286fb92744258cc1522647c347bb (127.10.114.67:34363)
I20260925 09:27:04.201179 14832 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:42373
I20260925 09:27:04.208521 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:27:04.225888 15022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.212828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56814 (local address 127.10.114.126:39829)
0925 09:27:04.212985 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:04.212989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:04.213460 (+ 471us) server_negotiation.cc:407] Connection header received
0925 09:27:04.213598 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:04.213601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:04.213659 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:04.213738 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:04.215003 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.215708 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.216939 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.217351 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.217442 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.220022 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:04.220031 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:04.220037 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:04.220077 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:04.223145 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:04.223824 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:04.223830 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:04.223834 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:04.223971 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:04.224335 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:04.224339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:04.224342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:04.224746 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:04.224979 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:04.225382 (+ 403us) server_negotiation.cc:299] Negotiation successful
0925 09:27:04.225603 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 09:27:04.229820 14832 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56814:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:27:04.236246 14832 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:27:04.251291 15020 heartbeater.cc:503] Master 127.10.114.126:39829 was elected leader, sending a full tablet report...
I20260925 09:27:04.264007 15298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.247244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42819 (local address 127.0.0.1:39680)
0925 09:27:04.248320 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:04.248363 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:04.248704 (+ 341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:04.255900 (+ 7196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:04.255906 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:04.255940 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:04.256384 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:04.256401 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.258572 (+ 2171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.258578 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:04.260933 (+ 2355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:04.260953 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.262938 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.262946 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:04.263203 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.263260 (+ 57us) client_negotiation.cc:769] Sending connection context
0925 09:27:04.263375 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 09:27:04.263559 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"mutex_wait_us":3,"thread_start_us":7809,"threads_started":1}
I20260925 09:27:04.264294 15297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.247231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39680 (local address 127.10.114.65:42819)
0925 09:27:04.255241 (+ 8010us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:04.255250 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:04.255277 (+ 27us) server_negotiation.cc:407] Connection header received
0925 09:27:04.255384 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:04.255394 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:04.255655 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:04.256615 (+ 960us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:04.256827 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.258407 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.261148 (+ 2741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.262715 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.263482 (+ 767us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.263707 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:04.263799 (+ 92us) server_negotiation.cc:299] Negotiation successful
0925 09:27:04.263955 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7853,"thread_start_us":7704,"threads_started":1}
I20260925 09:27:04.266167 15300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.247651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36412 (local address 127.10.114.67:34363)
0925 09:27:04.254905 (+ 7254us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:04.254918 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:04.255422 (+ 504us) server_negotiation.cc:407] Connection header received
0925 09:27:04.255695 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:04.255705 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:04.256966 (+ 1261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:04.257142 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:04.258048 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.259610 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.263595 (+ 3985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.265085 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.265204 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.265430 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:04.265701 (+ 271us) server_negotiation.cc:299] Negotiation successful
0925 09:27:04.265855 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7079,"thread_start_us":6956,"threads_started":1}
I20260925 09:27:04.266767 15299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.247467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:34363 (local address 127.0.0.1:36412)
0925 09:27:04.255317 (+ 7850us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:04.255336 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:04.255516 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:04.257425 (+ 1909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:04.257430 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:04.257452 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:04.257828 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:04.257840 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.259812 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.259818 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:04.263399 (+ 3581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:04.263414 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.265303 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.265310 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:04.265516 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.265546 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 09:27:04.266406 (+ 860us) client_negotiation.cc:240] Negotiation successful
0925 09:27:04.266515 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7712,"mutex_wait_us":375,"thread_start_us":399,"threads_started":1}
I20260925 09:27:04.268828 14955 tablet_service.cc:1467] Processing CreateTablet for tablet 47dfb896d6f440deb4748df37daeec3e (DEFAULT_TABLE table=test-table [id=aa3144dbfe6c40c1af39177664fb42ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:04.270290 14955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47dfb896d6f440deb4748df37daeec3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:04.270737 15302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.255683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33070 (local address 127.10.114.66:39291)
0925 09:27:04.263374 (+ 7691us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:04.263383 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:04.263643 (+ 260us) server_negotiation.cc:407] Connection header received
0925 09:27:04.264553 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:04.264563 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:04.264788 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:04.264970 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:04.265609 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.266971 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.268756 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.269719 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.269866 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.270042 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:04.270282 (+ 240us) server_negotiation.cc:299] Negotiation successful
0925 09:27:04.270426 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7542,"thread_start_us":7418,"threads_started":1}
I20260925 09:27:04.270740 15301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.255387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:39291 (local address 127.0.0.1:33070)
0925 09:27:04.263549 (+ 8162us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:04.263567 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:04.263886 (+ 319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:04.264974 (+ 1088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:04.264979 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:04.265008 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:04.265401 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:04.265411 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.267116 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.267121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:04.268608 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:04.268620 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.269922 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.269928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:04.270095 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.270126 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 09:27:04.270235 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 09:27:04.270427 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8013,"thread_start_us":8285,"threads_started":1}
I20260925 09:27:04.270270 15220 tablet_service.cc:1467] Processing CreateTablet for tablet 47dfb896d6f440deb4748df37daeec3e (DEFAULT_TABLE table=test-table [id=aa3144dbfe6c40c1af39177664fb42ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:04.271595 15220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47dfb896d6f440deb4748df37daeec3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:04.273627 15088 tablet_service.cc:1467] Processing CreateTablet for tablet 47dfb896d6f440deb4748df37daeec3e (DEFAULT_TABLE table=test-table [id=aa3144dbfe6c40c1af39177664fb42ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:04.274654 15088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47dfb896d6f440deb4748df37daeec3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:04.285063 15303 tablet_bootstrap.cc:492] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7: Bootstrap starting.
I20260925 09:27:04.288444 15304 tablet_bootstrap.cc:492] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb: Bootstrap starting.
I20260925 09:27:04.288823 15303 tablet_bootstrap.cc:654] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:04.290158 15303 log.cc:824] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:04.290282 15305 tablet_bootstrap.cc:492] T 47dfb896d6f440deb4748df37daeec3e P b0efc5d4d392485e803f05ed0678a418: Bootstrap starting.
I20260925 09:27:04.291740 15304 tablet_bootstrap.cc:654] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:04.292701 15304 log.cc:824] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:04.293848 15303 tablet_bootstrap.cc:492] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7: No bootstrap required, opened a new log
I20260925 09:27:04.293913 15305 tablet_bootstrap.cc:654] T 47dfb896d6f440deb4748df37daeec3e P b0efc5d4d392485e803f05ed0678a418: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:04.294108 15303 ts_tablet_manager.cc:1397] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260925 09:27:04.295154 15305 log.cc:824] T 47dfb896d6f440deb4748df37daeec3e P b0efc5d4d392485e803f05ed0678a418: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:04.296634 15304 tablet_bootstrap.cc:492] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb: No bootstrap required, opened a new log
I20260925 09:27:04.296913 15304 ts_tablet_manager.cc:1397] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260925 09:27:04.298332 15305 tablet_bootstrap.cc:492] T 47dfb896d6f440deb4748df37daeec3e P b0efc5d4d392485e803f05ed0678a418: No bootstrap required, opened a new log
I20260925 09:27:04.298607 15305 ts_tablet_manager.cc:1397] T 47dfb896d6f440deb4748df37daeec3e P b0efc5d4d392485e803f05ed0678a418: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.002s
I20260925 09:27:04.301843 15303 raft_consensus.cc:348] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0efc5d4d392485e803f05ed0678a418" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42819 } } peers { permanent_uuid: "324d286fb92744258cc1522647c347bb" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34363 } } peers { permanent_uuid: "3fc64ce9bc40435b8ded2190e1bb3af7" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39291 } }
I20260925 09:27:04.302186 15303 raft_consensus.cc:374] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:04.302310 15303 raft_consensus.cc:729] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc64ce9bc40435b8ded2190e1bb3af7, State: Initialized, Role: FOLLOWER
I20260925 09:27:04.302866 15303 consensus_queue.cc:260] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0efc5d4d392485e803f05ed0678a418" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42819 } } peers { permanent_uuid: "324d286fb92744258cc1522647c347bb" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34363 } } peers { permanent_uuid: "3fc64ce9bc40435b8ded2190e1bb3af7" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39291 } }
I20260925 09:27:04.303910 15153 heartbeater.cc:503] Master 127.10.114.126:39829 was elected leader, sending a full tablet report...
I20260925 09:27:04.304579 15303 ts_tablet_manager.cc:1428] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7: Time spent starting tablet: real 0.010s user 0.012s sys 0.000s
I20260925 09:27:04.307935 15304 raft_consensus.cc:348] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0efc5d4d392485e803f05ed0678a418" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42819 } } peers { permanent_uuid: "324d286fb92744258cc1522647c347bb" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34363 } } peers { permanent_uuid: "3fc64ce9bc40435b8ded2190e1bb3af7" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39291 } }
I20260925 09:27:04.308314 15304 raft_consensus.cc:374] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:04.308403 15304 raft_consensus.cc:729] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 324d286fb92744258cc1522647c347bb, State: Initialized, Role: FOLLOWER
I20260925 09:27:04.308941 15304 consensus_queue.cc:260] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0efc5d4d392485e803f05ed0678a418" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42819 } } peers { permanent_uuid: "324d286fb92744258cc1522647c347bb" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34363 } } peers { permanent_uuid: "3fc64ce9bc40435b8ded2190e1bb3af7" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39291 } }
I20260925 09:27:04.309432 15305 raft_consensus.cc:348] T 47dfb896d6f440deb4748df37daeec3e P b0efc5d4d392485e803f05ed0678a418 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0efc5d4d392485e803f05ed0678a418" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42819 } } peers { permanent_uuid: "324d286fb92744258cc1522647c347bb" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34363 } } peers { permanent_uuid: "3fc64ce9bc40435b8ded2190e1bb3af7" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39291 } }
I20260925 09:27:04.309852 15305 raft_consensus.cc:374] T 47dfb896d6f440deb4748df37daeec3e P b0efc5d4d392485e803f05ed0678a418 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:04.309942 15305 raft_consensus.cc:729] T 47dfb896d6f440deb4748df37daeec3e P b0efc5d4d392485e803f05ed0678a418 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0efc5d4d392485e803f05ed0678a418, State: Initialized, Role: FOLLOWER
I20260925 09:27:04.310156 15285 heartbeater.cc:503] Master 127.10.114.126:39829 was elected leader, sending a full tablet report...
I20260925 09:27:04.310539 15305 consensus_queue.cc:260] T 47dfb896d6f440deb4748df37daeec3e P b0efc5d4d392485e803f05ed0678a418 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0efc5d4d392485e803f05ed0678a418" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42819 } } peers { permanent_uuid: "324d286fb92744258cc1522647c347bb" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34363 } } peers { permanent_uuid: "3fc64ce9bc40435b8ded2190e1bb3af7" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39291 } }
I20260925 09:27:04.310765 15304 ts_tablet_manager.cc:1428] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb: Time spent starting tablet: real 0.014s user 0.011s sys 0.003s
I20260925 09:27:04.312212 15305 ts_tablet_manager.cc:1428] T 47dfb896d6f440deb4748df37daeec3e P b0efc5d4d392485e803f05ed0678a418: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260925 09:27:04.357105 15309 raft_consensus.cc:482] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:27:04.357370 15309 raft_consensus.cc:504] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0efc5d4d392485e803f05ed0678a418" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42819 } } peers { permanent_uuid: "324d286fb92744258cc1522647c347bb" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34363 } } peers { permanent_uuid: "3fc64ce9bc40435b8ded2190e1bb3af7" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39291 } }
I20260925 09:27:04.357811 15311 raft_consensus.cc:482] T 47dfb896d6f440deb4748df37daeec3e P b0efc5d4d392485e803f05ed0678a418 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:27:04.358062 15311 raft_consensus.cc:504] T 47dfb896d6f440deb4748df37daeec3e P b0efc5d4d392485e803f05ed0678a418 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0efc5d4d392485e803f05ed0678a418" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42819 } } peers { permanent_uuid: "324d286fb92744258cc1522647c347bb" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34363 } } peers { permanent_uuid: "3fc64ce9bc40435b8ded2190e1bb3af7" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39291 } }
I20260925 09:27:04.359195 15309 leader_election.cc:288] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0efc5d4d392485e803f05ed0678a418 (127.10.114.65:42819), 324d286fb92744258cc1522647c347bb (127.10.114.67:34363)
I20260925 09:27:04.359628 15311 leader_election.cc:288] T 47dfb896d6f440deb4748df37daeec3e P b0efc5d4d392485e803f05ed0678a418 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 324d286fb92744258cc1522647c347bb (127.10.114.67:34363), 3fc64ce9bc40435b8ded2190e1bb3af7 (127.10.114.66:39291)
I20260925 09:27:04.371187 15300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.359812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:56503 (local address 127.10.114.67:34363)
0925 09:27:04.360075 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:04.360083 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:04.362177 (+ 2094us) server_negotiation.cc:407] Connection header received
0925 09:27:04.362472 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:04.362480 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:04.362587 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:04.362661 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:04.363516 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.365008 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.368233 (+ 3225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.369669 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.369788 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.369871 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:04.370629 (+ 758us) server_negotiation.cc:299] Negotiation successful
0925 09:27:04.370807 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260925 09:27:04.372283 15313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.359672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:34363 (local address 127.10.114.66:56503)
0925 09:27:04.362075 (+ 2403us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:04.362096 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:04.362299 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:04.362886 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:04.362894 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:04.362925 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:04.363312 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:04.363326 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.365230 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.365238 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:04.368007 (+ 2769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:04.368026 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.370092 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.370099 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:04.370365 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.370418 (+ 53us) client_negotiation.cc:769] Sending connection context
0925 09:27:04.371680 (+ 1262us) client_negotiation.cc:240] Negotiation successful
0925 09:27:04.371892 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2285,"thread_start_us":2099,"threads_started":1}
I20260925 09:27:04.374086 15240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47dfb896d6f440deb4748df37daeec3e" candidate_uuid: "3fc64ce9bc40435b8ded2190e1bb3af7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "324d286fb92744258cc1522647c347bb" is_pre_election: true
I20260925 09:27:04.374358 15302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.362061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:40307 (local address 127.10.114.66:39291)
0925 09:27:04.362236 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:04.362244 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:04.362267 (+ 23us) server_negotiation.cc:407] Connection header received
0925 09:27:04.362338 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:04.362346 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:04.362434 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:04.362541 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:04.363481 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.364891 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.372325 (+ 7434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.373310 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.373466 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.373540 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:04.373996 (+ 456us) server_negotiation.cc:299] Negotiation successful
0925 09:27:04.374133 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 09:27:04.374634 15240 raft_consensus.cc:2454] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fc64ce9bc40435b8ded2190e1bb3af7 in term 0.
I20260925 09:27:04.375082 15315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.359841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:39291 (local address 127.10.114.65:40307)
0925 09:27:04.361312 (+ 1471us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:04.361340 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:04.361481 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:04.362743 (+ 1262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:04.362751 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:04.362786 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:04.363287 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:04.363302 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.365064 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.365070 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:04.368027 (+ 2957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:04.368047 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.373570 (+ 5523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.373579 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:04.373802 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.373842 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 09:27:04.374489 (+ 647us) client_negotiation.cc:240] Negotiation successful
0925 09:27:04.374658 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1300,"mutex_wait_us":9,"thread_start_us":12233,"threads_started":1}
I20260925 09:27:04.375357 15042 leader_election.cc:302] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 324d286fb92744258cc1522647c347bb, 3fc64ce9bc40435b8ded2190e1bb3af7; no voters:
I20260925 09:27:04.375732 15309 raft_consensus.cc:2781] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:27:04.375900 15309 raft_consensus.cc:482] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:27:04.376024 15309 raft_consensus.cc:3037] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:04.376281 15297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.361328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:43679 (local address 127.10.114.65:42819)
0925 09:27:04.361561 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:04.361570 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:04.361593 (+ 23us) server_negotiation.cc:407] Connection header received
0925 09:27:04.361665 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:04.361672 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:04.361763 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:04.361861 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:04.365129 (+ 3268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.369576 (+ 4447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.372637 (+ 3061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.374013 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.374126 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.374198 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:04.375659 (+ 1461us) server_negotiation.cc:299] Negotiation successful
0925 09:27:04.375813 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3040,"server-negotiator.queue_time_us":67}
I20260925 09:27:04.376919 15312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.359356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42819 (local address 127.10.114.66:43679)
0925 09:27:04.360465 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:04.360501 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:04.360666 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:04.364007 (+ 3341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:04.364012 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:04.364035 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:04.364430 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:04.364440 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.369784 (+ 5344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.369793 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:04.372304 (+ 2511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:04.372321 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.375117 (+ 2796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.375126 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:04.375497 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.375532 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 09:27:04.375618 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 09:27:04.375764 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":929,"thread_start_us":725,"threads_started":1}
I20260925 09:27:04.376685 15108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47dfb896d6f440deb4748df37daeec3e" candidate_uuid: "b0efc5d4d392485e803f05ed0678a418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fc64ce9bc40435b8ded2190e1bb3af7" is_pre_election: true
I20260925 09:27:04.379117 14975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47dfb896d6f440deb4748df37daeec3e" candidate_uuid: "3fc64ce9bc40435b8ded2190e1bb3af7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0efc5d4d392485e803f05ed0678a418" is_pre_election: true
I20260925 09:27:04.379390 15300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.360795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:41587 (local address 127.10.114.67:34363)
0925 09:27:04.371460 (+ 10665us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:04.371468 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:04.372619 (+ 1151us) server_negotiation.cc:407] Connection header received
0925 09:27:04.372851 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:04.372857 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:04.372920 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:04.372974 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:04.373771 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.374944 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.377344 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.378486 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.378614 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.378673 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:04.378982 (+ 309us) server_negotiation.cc:299] Negotiation successful
0925 09:27:04.379097 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10534,"thread_start_us":11062,"threads_started":1}
I20260925 09:27:04.379374 15309 raft_consensus.cc:504] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0efc5d4d392485e803f05ed0678a418" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42819 } } peers { permanent_uuid: "324d286fb92744258cc1522647c347bb" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34363 } } peers { permanent_uuid: "3fc64ce9bc40435b8ded2190e1bb3af7" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39291 } }
I20260925 09:27:04.379573 15314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.360264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:34363 (local address 127.10.114.65:41587)
0925 09:27:04.372534 (+ 12270us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:04.372553 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:04.372708 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:04.373136 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:04.373140 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:04.373173 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:04.373559 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:04.373571 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.375096 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.375103 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:04.377169 (+ 2066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:04.377185 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.378648 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.378652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:04.378760 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.378780 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 09:27:04.378959 (+ 179us) client_negotiation.cc:240] Negotiation successful
0925 09:27:04.379091 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11968,"mutex_wait_us":1,"thread_start_us":1729,"threads_started":1}
I20260925 09:27:04.379611 14975 raft_consensus.cc:2454] T 47dfb896d6f440deb4748df37daeec3e P b0efc5d4d392485e803f05ed0678a418 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fc64ce9bc40435b8ded2190e1bb3af7 in term 0.
I20260925 09:27:04.379801 15108 raft_consensus.cc:2379] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b0efc5d4d392485e803f05ed0678a418 in current term 1: Already voted for candidate 3fc64ce9bc40435b8ded2190e1bb3af7 in this term.
I20260925 09:27:04.380317 15240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47dfb896d6f440deb4748df37daeec3e" candidate_uuid: "b0efc5d4d392485e803f05ed0678a418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "324d286fb92744258cc1522647c347bb" is_pre_election: true
I20260925 09:27:04.380519 15240 raft_consensus.cc:2454] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0efc5d4d392485e803f05ed0678a418 in term 0.
I20260925 09:27:04.380738 15309 leader_election.cc:288] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [CANDIDATE]: Term 1 election: Requested vote from peers b0efc5d4d392485e803f05ed0678a418 (127.10.114.65:42819), 324d286fb92744258cc1522647c347bb (127.10.114.67:34363)
I20260925 09:27:04.381007 14909 leader_election.cc:302] T 47dfb896d6f440deb4748df37daeec3e P b0efc5d4d392485e803f05ed0678a418 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 324d286fb92744258cc1522647c347bb, b0efc5d4d392485e803f05ed0678a418; no voters:
I20260925 09:27:04.381219 14975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47dfb896d6f440deb4748df37daeec3e" candidate_uuid: "3fc64ce9bc40435b8ded2190e1bb3af7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0efc5d4d392485e803f05ed0678a418"
I20260925 09:27:04.381412 14975 raft_consensus.cc:3037] T 47dfb896d6f440deb4748df37daeec3e P b0efc5d4d392485e803f05ed0678a418 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:04.381376 15240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47dfb896d6f440deb4748df37daeec3e" candidate_uuid: "3fc64ce9bc40435b8ded2190e1bb3af7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "324d286fb92744258cc1522647c347bb"
I20260925 09:27:04.381556 15240 raft_consensus.cc:3037] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:04.384545 14975 raft_consensus.cc:2454] T 47dfb896d6f440deb4748df37daeec3e P b0efc5d4d392485e803f05ed0678a418 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc64ce9bc40435b8ded2190e1bb3af7 in term 1.
I20260925 09:27:04.384574 15240 raft_consensus.cc:2454] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc64ce9bc40435b8ded2190e1bb3af7 in term 1.
I20260925 09:27:04.384768 15311 raft_consensus.cc:2741] T 47dfb896d6f440deb4748df37daeec3e P b0efc5d4d392485e803f05ed0678a418 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260925 09:27:04.384938 15042 leader_election.cc:302] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 324d286fb92744258cc1522647c347bb, 3fc64ce9bc40435b8ded2190e1bb3af7; no voters:
I20260925 09:27:04.385217 15309 raft_consensus.cc:2781] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:04.385684 15309 raft_consensus.cc:686] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [term 1 LEADER]: Becoming Leader. State: Replica: 3fc64ce9bc40435b8ded2190e1bb3af7, State: Running, Role: LEADER
I20260925 09:27:04.386155 15309 consensus_queue.cc:237] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0efc5d4d392485e803f05ed0678a418" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42819 } } peers { permanent_uuid: "324d286fb92744258cc1522647c347bb" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34363 } } peers { permanent_uuid: "3fc64ce9bc40435b8ded2190e1bb3af7" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39291 } }
I20260925 09:27:04.390057 14831 catalog_manager.cc:5495] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fc64ce9bc40435b8ded2190e1bb3af7 (127.10.114.66). New cstate: current_term: 1 leader_uuid: "3fc64ce9bc40435b8ded2190e1bb3af7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0efc5d4d392485e803f05ed0678a418" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "324d286fb92744258cc1522647c347bb" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fc64ce9bc40435b8ded2190e1bb3af7" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39291 } health_report { overall_health: HEALTHY } } }
I20260925 09:27:04.411026 15302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.405489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33076 (local address 127.10.114.66:39291)
0925 09:27:04.405685 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:04.405691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:04.405710 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:27:04.405925 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:04.405931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:04.406015 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:04.406184 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:27:04.406571 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.407335 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.408992 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.409406 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.409578 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.410048 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:27:04.410494 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:27:04.410626 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:04.410734 (+ 108us) server_negotiation.cc:299] Negotiation successful
0925 09:27:04.410794 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:27:04.418397 15240 raft_consensus.cc:1261] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb [term 1 FOLLOWER]: Refusing update from remote peer 3fc64ce9bc40435b8ded2190e1bb3af7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:27:04.418433 14975 raft_consensus.cc:1261] T 47dfb896d6f440deb4748df37daeec3e P b0efc5d4d392485e803f05ed0678a418 [term 1 FOLLOWER]: Refusing update from remote peer 3fc64ce9bc40435b8ded2190e1bb3af7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:27:04.419205 15317 consensus_queue.cc:1035] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "324d286fb92744258cc1522647c347bb" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:27:04.419622 15309 consensus_queue.cc:1035] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0efc5d4d392485e803f05ed0678a418" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:27:04.426731 15286 tablet.cc:2369] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:27:04.433779 15320 mvcc.cc:205] Tried to move back new op lower bound from 7333185226408906752 to 7333185226289041408. Current Snapshot: MvccSnapshot[applied={T|T < 7333185226408906752}]
I20260925 09:27:04.444268 14831 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:27:04.444548 14831 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:27:04.447878 14831 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfef81902a7f47e49cba9d299b83ed64: Sending DeleteTablet for 3 replicas of tablet 47dfb896d6f440deb4748df37daeec3e
I20260925 09:27:04.448920 14955 tablet_service.cc:1514] Processing DeleteTablet for tablet 47dfb896d6f440deb4748df37daeec3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:04 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:39680
I20260925 09:27:04.448908 15220 tablet_service.cc:1514] Processing DeleteTablet for tablet 47dfb896d6f440deb4748df37daeec3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:04 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:36412
I20260925 09:27:04.448971 15088 tablet_service.cc:1514] Processing DeleteTablet for tablet 47dfb896d6f440deb4748df37daeec3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:04 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:33070
I20260925 09:27:04.453738 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 14893
I20260925 09:27:04.471479 15331 tablet_replica.cc:330] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7: stopping tablet replica
I20260925 09:27:04.471954 15331 raft_consensus.cc:2229] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:27:04.472597 15331 raft_consensus.cc:2258] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:04.475559 15331 ts_tablet_manager.cc:1905] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:04.482378 15330 tablet_replica.cc:330] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb: stopping tablet replica
I20260925 09:27:04.482789 15330 raft_consensus.cc:2229] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:27:04.483299 15330 raft_consensus.cc:2258] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb [term 1 FOLLOWER]: Raft consensus is shut down!
Sep 25 09:27:04 dist-test-slave-9pj8 krb5kdc[14788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:27:04.485936 15330 ts_tablet_manager.cc:1905] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:04.492542 15330 ts_tablet_manager.cc:1918] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:27:04.492730 15330 log.cc:1196] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/47dfb896d6f440deb4748df37daeec3e
I20260925 09:27:04.493242 15330 ts_tablet_manager.cc:1939] T 47dfb896d6f440deb4748df37daeec3e P 324d286fb92744258cc1522647c347bb: Deleting consensus metadata
I20260925 09:27:04.495052 14817 catalog_manager.cc:4841] TS 324d286fb92744258cc1522647c347bb (127.10.114.67:34363): tablet 47dfb896d6f440deb4748df37daeec3e (table test-table [id=aa3144dbfe6c40c1af39177664fb42ff]) successfully deleted
I20260925 09:27:04.496943 15297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.455391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39684 (local address 127.10.114.65:42819)
0925 09:27:04.455608 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:04.455617 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:04.460177 (+ 4560us) server_negotiation.cc:407] Connection header received
0925 09:27:04.471434 (+ 11257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:04.471443 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:04.471563 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:04.471696 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:04.473883 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.475482 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.478081 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.479003 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.479158 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.486555 (+ 7397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:04.486575 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:04.486599 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:04.486654 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:04.491640 (+ 4986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:04.492803 (+ 1163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:04.492858 (+ 55us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:04.492864 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:04.493086 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:04.493679 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:04.493686 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:04.493692 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:04.494226 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:04.494614 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:04.496070 (+ 1456us) server_negotiation.cc:299] Negotiation successful
0925 09:27:04.496367 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:27:04.498230 15331 ts_tablet_manager.cc:1918] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:27:04.498471 15331 log.cc:1196] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/47dfb896d6f440deb4748df37daeec3e
I20260925 09:27:04.499346 15331 ts_tablet_manager.cc:1939] T 47dfb896d6f440deb4748df37daeec3e P 3fc64ce9bc40435b8ded2190e1bb3af7: Deleting consensus metadata
I20260925 09:27:04.501727 14819 catalog_manager.cc:4841] TS 3fc64ce9bc40435b8ded2190e1bb3af7 (127.10.114.66:39291): tablet 47dfb896d6f440deb4748df37daeec3e (table test-table [id=aa3144dbfe6c40c1af39177664fb42ff]) successfully deleted
I20260925 09:27:04.502346 15015 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:04.595156 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 14893
W20260925 09:27:04.619838 14819 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.114.65:42819 (error 108)
I20260925 09:27:04.620569 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 15026
I20260925 09:27:04.621919 15301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.621471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42819 (local address 127.0.0.1:39700)
0925 09:27:04.621709 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:04.621794 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:42819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
W20260925 09:27:04.622228 14819 catalog_manager.cc:4584] TS b0efc5d4d392485e803f05ed0678a418 (127.10.114.65:42819): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 47dfb896d6f440deb4748df37daeec3e: Network error: Client connection negotiation failed: client connection to 127.10.114.65:42819: connect: Connection refused (error 111)
Sep 25 09:27:04 dist-test-slave-9pj8 krb5kdc[14788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.66@KRBTEST.COM
I20260925 09:27:04.637712 15302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.621364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33092 (local address 127.10.114.66:39291)
0925 09:27:04.621564 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:04.621570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:04.621836 (+ 266us) server_negotiation.cc:407] Connection header received
0925 09:27:04.621904 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:04.621910 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:04.621998 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:04.622088 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:04.623885 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.624722 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.626509 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.626931 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.627025 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.631674 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:04.631686 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:04.631700 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:04.631732 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:04.634893 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:04.635557 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:04.635565 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:04.635570 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:04.635703 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:04.636106 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:04.636110 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:04.636112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:04.636457 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:04.636789 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:04.637088 (+ 299us) server_negotiation.cc:299] Negotiation successful
0925 09:27:04.637365 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:27:04.638314 15148 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:04.675843 15301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.675393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42819 (local address 127.0.0.1:39714)
0925 09:27:04.675574 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:04.675662 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:42819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260925 09:27:04.676748 15301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.676476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42819 (local address 127.0.0.1:39716)
0925 09:27:04.676593 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:04.676651 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:42819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 09:27:04.737365 15301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.736866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42819 (local address 127.0.0.1:39726)
0925 09:27:04.737076 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:04.737196 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:42819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260925 09:27:04.738307 15301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.738025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42819 (local address 127.0.0.1:39736)
0925 09:27:04.738154 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:04.738214 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:42819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260925 09:27:04.751420 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 15026
I20260925 09:27:04.769397 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 15158
Sep 25 09:27:04 dist-test-slave-9pj8 krb5kdc[14788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.67@KRBTEST.COM
I20260925 09:27:04.782866 15300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.770129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36428 (local address 127.10.114.67:34363)
0925 09:27:04.770301 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:04.770306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:04.770323 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:27:04.770443 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:04.770447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:04.770528 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:04.770611 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:04.771759 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.772464 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.773599 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:04.773967 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:04.774052 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:04.778145 (+ 4093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:04.778159 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:04.778175 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:04.778205 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:04.780746 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:04.781290 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:04.781297 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:04.781299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:04.781412 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:04.781721 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:04.781724 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:04.781726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:04.782053 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:04.782257 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:04.782498 (+ 241us) server_negotiation.cc:299] Negotiation successful
0925 09:27:04.782652 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 09:27:04.783488 15280 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:04.819201 15301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.818740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42819 (local address 127.0.0.1:39740)
0925 09:27:04.818963 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:04.819054 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:42819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260925 09:27:04.820304 15301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.819897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42819 (local address 127.0.0.1:39746)
0925 09:27:04.820119 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:04.820196 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:42819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260925 09:27:04.871534 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 15158
I20260925 09:27:04.888341 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 14801
I20260925 09:27:04.889583 14862 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:04.995527 15301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:04.994835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42819 (local address 127.0.0.1:39756)
0925 09:27:04.995174 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:04.995286 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:42819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260925 09:27:04.995601 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 14801
2026-09-25T09:27:05Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4007 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-25T09:27:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:27:05Z Disabled control of system clock
I20260925 09:27:05.060979 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:39891
--webserver_interface=127.10.114.126
--webserver_port=0
--builtin_ntp_servers=127.10.114.84:43713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:39891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 09:27:05.364107 15341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:05.364362 15341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:05.364437 15341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:05.370971 15341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:27:05.371053 15341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:05.371093 15341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:05.371145 15341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:27:05.371186 15341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:27:05.380245 15341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:43713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:39891
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:39891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:05.381311 15341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:05.382887 15341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:05.394047 15346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:05.394085 15347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:05.394066 15349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:05.394959 15341 server_base.cc:1034] running on GCE node
I20260925 09:27:05.395695 15341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:05.397035 15341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:05.398267 15341 hybrid_clock.cc:648] HybridClock initialized: now 1790328425398226 us; error 56 us; skew 500 ppm
I20260925 09:27:05.398590 15341 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:27:05.400839 15341 webserver.cc:460] Webserver started at http://127.10.114.126:45829/ using document root <none> and password file <none>
I20260925 09:27:05.401501 15341 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:05.401628 15341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:05.401918 15341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:05.404583 15341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "12d0c8f2d5f44a8aa4f23cff11604c5f"
format_stamp: "Formatted at 2026-09-25 09:27:05 on dist-test-slave-9pj8"
server_key: "27aaccfdc1173cc1fd1549f37595e730"
server_key_iv: "1d4375fb9ebb3993c29404c12338b59c"
server_key_version: "encryptionkey@0"
I20260925 09:27:05.405323 15341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "12d0c8f2d5f44a8aa4f23cff11604c5f"
format_stamp: "Formatted at 2026-09-25 09:27:05 on dist-test-slave-9pj8"
server_key: "27aaccfdc1173cc1fd1549f37595e730"
server_key_iv: "1d4375fb9ebb3993c29404c12338b59c"
server_key_version: "encryptionkey@0"
I20260925 09:27:05.409914 15341 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260925 09:27:05.412982 15355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:05.414222 15341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:27:05.414386 15341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "12d0c8f2d5f44a8aa4f23cff11604c5f"
format_stamp: "Formatted at 2026-09-25 09:27:05 on dist-test-slave-9pj8"
server_key: "27aaccfdc1173cc1fd1549f37595e730"
server_key_iv: "1d4375fb9ebb3993c29404c12338b59c"
server_key_version: "encryptionkey@0"
I20260925 09:27:05.414582 15341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:05.420012 15341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:05.420687 15341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:05.420981 15341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:05.438625 15341 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:39891
I20260925 09:27:05.438699 15406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:39891 every 8 connection(s)
I20260925 09:27:05.439894 15341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:27:05.443207 15407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:05.445637 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 15341
I20260925 09:27:05.445845 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:27:05.446456 10697 external_mini_cluster.cc:1428] Setting key 0d80e6d7eb3d16ebd73f63d95fbfcd1a
I20260925 09:27:05.455376 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:05.448476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60128 (local address 127.10.114.126:39891)
0925 09:27:05.449454 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:05.449470 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:05.449519 (+ 49us) server_negotiation.cc:407] Connection header received
0925 09:27:05.451571 (+ 2052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:05.451602 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:05.452147 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:05.452559 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:05.453539 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:05.453546 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:05.453561 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:05.453577 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:05.453654 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:05.453673 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:05.453884 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:05.454027 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 09:27:05.454277 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":314,"threads_started":1}
I20260925 09:27:05.456115 15407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f: Bootstrap starting.
I20260925 09:27:05.459035 15407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:05.460017 15407 log.cc:824] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:05.462358 15407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f: No bootstrap required, opened a new log
I20260925 09:27:05.469079 15407 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0c8f2d5f44a8aa4f23cff11604c5f" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39891 } }
I20260925 09:27:05.469376 15407 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:05.469441 15407 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12d0c8f2d5f44a8aa4f23cff11604c5f, State: Initialized, Role: FOLLOWER
I20260925 09:27:05.469871 15407 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0c8f2d5f44a8aa4f23cff11604c5f" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39891 } }
I20260925 09:27:05.470043 15407 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:27:05.470127 15407 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:27:05.470228 15407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:05.472445 15407 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0c8f2d5f44a8aa4f23cff11604c5f" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39891 } }
I20260925 09:27:05.472854 15407 leader_election.cc:302] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12d0c8f2d5f44a8aa4f23cff11604c5f; no voters:
I20260925 09:27:05.473349 15407 leader_election.cc:288] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:27:05.473491 15412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:05.474197 15412 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f [term 1 LEADER]: Becoming Leader. State: Replica: 12d0c8f2d5f44a8aa4f23cff11604c5f, State: Running, Role: LEADER
I20260925 09:27:05.474743 15412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0c8f2d5f44a8aa4f23cff11604c5f" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39891 } }
I20260925 09:27:05.475291 15407 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:27:05.478096 15413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12d0c8f2d5f44a8aa4f23cff11604c5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0c8f2d5f44a8aa4f23cff11604c5f" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39891 } } }
I20260925 09:27:05.478503 15413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:05.479157 15414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12d0c8f2d5f44a8aa4f23cff11604c5f. Latest consensus state: current_term: 1 leader_uuid: "12d0c8f2d5f44a8aa4f23cff11604c5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0c8f2d5f44a8aa4f23cff11604c5f" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39891 } } }
I20260925 09:27:05.479978 15414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:05.481973 15421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:27:05.486725 15421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:27:05.495966 15421 catalog_manager.cc:1347] Generated new cluster ID: 5b43674d84954ea9bdc8bb3cb35cfb23
I20260925 09:27:05.496079 15421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:27:05.507428 15421 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:27:05.507825 15421 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:27:05.518812 15421 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f: Generated new TSK 0
I20260925 09:27:05.519567 15421 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:27:05.527529 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39891
--builtin_ntp_servers=127.10.114.84:43713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 09:27:05.870667 15431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:05.870934 15431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:05.871019 15431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:05.878185 15431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:05.878274 15431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:05.878378 15431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:27:05.887159 15431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:43713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:05.888458 15431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:05.890092 15431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:05.902222 15439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:05.902256 15437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:05.902307 15436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:05.903334 15431 server_base.cc:1034] running on GCE node
I20260925 09:27:05.903977 15431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:05.904829 15431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:05.906013 15431 hybrid_clock.cc:648] HybridClock initialized: now 1790328425905956 us; error 49 us; skew 500 ppm
I20260925 09:27:05.906309 15431 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:27:05.908382 15431 webserver.cc:460] Webserver started at http://127.10.114.65:34871/ using document root <none> and password file <none>
I20260925 09:27:05.909011 15431 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:05.909126 15431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:05.909441 15431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:05.912065 15431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "325484a91d9e414bac6d6fd1167da87d"
format_stamp: "Formatted at 2026-09-25 09:27:05 on dist-test-slave-9pj8"
server_key: "57b8ccc294622abe1be998e9b34277f9"
server_key_iv: "3d347cbb15603dba24d5a574edb94da7"
server_key_version: "encryptionkey@0"
I20260925 09:27:05.912864 15431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "325484a91d9e414bac6d6fd1167da87d"
format_stamp: "Formatted at 2026-09-25 09:27:05 on dist-test-slave-9pj8"
server_key: "57b8ccc294622abe1be998e9b34277f9"
server_key_iv: "3d347cbb15603dba24d5a574edb94da7"
server_key_version: "encryptionkey@0"
I20260925 09:27:05.917479 15431 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:27:05.920742 15445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:05.921921 15431 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 09:27:05.922077 15431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "325484a91d9e414bac6d6fd1167da87d"
format_stamp: "Formatted at 2026-09-25 09:27:05 on dist-test-slave-9pj8"
server_key: "57b8ccc294622abe1be998e9b34277f9"
server_key_iv: "3d347cbb15603dba24d5a574edb94da7"
server_key_version: "encryptionkey@0"
I20260925 09:27:05.922286 15431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:05.927932 15431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:05.928675 15431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:05.929003 15431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:05.929850 15431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:05.931140 15431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:05.931267 15431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:05.931397 15431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:05.931485 15431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:05.962602 15431 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:42307
I20260925 09:27:05.962656 15557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:42307 every 8 connection(s)
I20260925 09:27:05.963899 15431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:27:05.967742 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 15431
I20260925 09:27:05.967948 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:27:05.968767 10697 external_mini_cluster.cc:1428] Setting key 7d92e6e8be48009431c3b2c399685dd3
I20260925 09:27:05.971688 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39891
--builtin_ntp_servers=127.10.114.84:43713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260925 09:27:05.972561 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:05.966235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:41933 (local address 127.10.114.126:39891)
0925 09:27:05.966621 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:05.966626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:05.968083 (+ 1457us) server_negotiation.cc:407] Connection header received
0925 09:27:05.970200 (+ 2117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:05.970206 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:05.970282 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:05.970399 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:05.971862 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:05.971871 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:05.971877 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:05.971902 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:05.971956 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:05.971968 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:05.972069 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:05.972298 (+ 229us) server_negotiation.cc:299] Negotiation successful
0925 09:27:05.972342 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":192,"threads_started":1}
I20260925 09:27:05.973770 15561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:05.966823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39891 (local address 127.10.114.65:41933)
0925 09:27:05.967862 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:05.967919 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:05.969937 (+ 2018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:05.970659 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:05.970670 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:05.970784 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:05.971463 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:27:05.971468 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:05.971520 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:27:05.971626 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:05.972074 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:05.972086 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:05.972097 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 09:27:05.972374 (+ 277us) client_negotiation.cc:240] Negotiation successful
0925 09:27:05.972614 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":246,"threads_started":1}
I20260925 09:27:05.975121 15558 heartbeater.cc:347] Connected to a master server at 127.10.114.126:39891
I20260925 09:27:05.975428 15558 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:05.976138 15558 heartbeater.cc:511] Master 127.10.114.126:39891 requested a full tablet report, sending...
I20260925 09:27:05.978050 15372 ts_manager.cc:194] Registered new tserver with Master: 325484a91d9e414bac6d6fd1167da87d (127.10.114.65:42307)
W20260925 09:27:06.284128 15562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:06.284399 15562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:06.284487 15562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:06.291389 15562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:06.291474 15562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:06.291574 15562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:27:06.300282 15562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:43713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:06.301441 15562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:06.303076 15562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:06.315721 15567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:06.315655 15568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:06.315698 15570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:06.316396 15562 server_base.cc:1034] running on GCE node
I20260925 09:27:06.316916 15562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:06.317822 15562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:06.319008 15562 hybrid_clock.cc:648] HybridClock initialized: now 1790328426318954 us; error 49 us; skew 500 ppm
I20260925 09:27:06.319281 15562 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:27:06.321382 15562 webserver.cc:460] Webserver started at http://127.10.114.66:43945/ using document root <none> and password file <none>
I20260925 09:27:06.321990 15562 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:06.322070 15562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:06.322314 15562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:06.324925 15562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "b435ced4718b442298b7880433fb7916"
format_stamp: "Formatted at 2026-09-25 09:27:06 on dist-test-slave-9pj8"
server_key: "da8e32ede3a60840982651eea451fe14"
server_key_iv: "43584fa7a3b91fdeed53b1a050b090d7"
server_key_version: "encryptionkey@0"
I20260925 09:27:06.325786 15562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "b435ced4718b442298b7880433fb7916"
format_stamp: "Formatted at 2026-09-25 09:27:06 on dist-test-slave-9pj8"
server_key: "da8e32ede3a60840982651eea451fe14"
server_key_iv: "43584fa7a3b91fdeed53b1a050b090d7"
server_key_version: "encryptionkey@0"
I20260925 09:27:06.330547 15562 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 09:27:06.333632 15576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:06.334824 15562 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 09:27:06.334944 15562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "b435ced4718b442298b7880433fb7916"
format_stamp: "Formatted at 2026-09-25 09:27:06 on dist-test-slave-9pj8"
server_key: "da8e32ede3a60840982651eea451fe14"
server_key_iv: "43584fa7a3b91fdeed53b1a050b090d7"
server_key_version: "encryptionkey@0"
I20260925 09:27:06.335103 15562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:06.340425 15562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:06.341086 15562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:06.341301 15562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:06.342080 15562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:06.343313 15562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:06.343395 15562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:06.343472 15562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:06.343524 15562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:06.376459 15562 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:40983
I20260925 09:27:06.376513 15688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:40983 every 8 connection(s)
I20260925 09:27:06.377785 15562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:27:06.380987 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 15562
I20260925 09:27:06.381155 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:27:06.381754 10697 external_mini_cluster.cc:1428] Setting key f0a418c7c98c226ab20c7bc48e7bd43e
I20260925 09:27:06.384521 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39891
--builtin_ntp_servers=127.10.114.84:43713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260925 09:27:06.386794 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:06.380517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:50987 (local address 127.10.114.126:39891)
0925 09:27:06.380711 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:06.380715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:06.382415 (+ 1700us) server_negotiation.cc:407] Connection header received
0925 09:27:06.384520 (+ 2105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:06.384525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:06.384603 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:06.384711 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:06.386106 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:06.386111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:06.386116 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:06.386134 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:06.386173 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:06.386184 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:06.386304 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:06.386500 (+ 196us) server_negotiation.cc:299] Negotiation successful
0925 09:27:06.386564 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260925 09:27:06.387658 15691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:06.381193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39891 (local address 127.10.114.66:50987)
0925 09:27:06.382216 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:06.382252 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:06.384267 (+ 2015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:06.385027 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:06.385035 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:06.385121 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:06.385796 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:27:06.385799 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:06.385841 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:27:06.385927 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:06.386284 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:06.386294 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:06.386301 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 09:27:06.386574 (+ 273us) client_negotiation.cc:240] Negotiation successful
0925 09:27:06.386801 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":317,"threads_started":1}
I20260925 09:27:06.388998 15689 heartbeater.cc:347] Connected to a master server at 127.10.114.126:39891
I20260925 09:27:06.389277 15689 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:06.389811 15689 heartbeater.cc:511] Master 127.10.114.126:39891 requested a full tablet report, sending...
I20260925 09:27:06.391198 15372 ts_manager.cc:194] Registered new tserver with Master: b435ced4718b442298b7880433fb7916 (127.10.114.66:40983)
W20260925 09:27:06.698316 15692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:06.698558 15692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:06.698716 15692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:06.705765 15692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:06.705847 15692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:06.706022 15692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:27:06.714670 15692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:43713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:06.715898 15692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:06.717583 15692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:06.730535 15700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:06.730810 15697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:06.730602 15698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:06.731824 15692 server_base.cc:1034] running on GCE node
I20260925 09:27:06.732390 15692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:06.733268 15692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:06.734465 15692 hybrid_clock.cc:648] HybridClock initialized: now 1790328426734418 us; error 24 us; skew 500 ppm
I20260925 09:27:06.734757 15692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:27:06.736860 15692 webserver.cc:460] Webserver started at http://127.10.114.67:39311/ using document root <none> and password file <none>
I20260925 09:27:06.737543 15692 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:06.737659 15692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:06.738003 15692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:06.740792 15692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "8d3c95b77ace4ee496b2745a4d6330d8"
format_stamp: "Formatted at 2026-09-25 09:27:06 on dist-test-slave-9pj8"
server_key: "30182f188322059a07d18a292157f02a"
server_key_iv: "ab3f32d38c077ac2a2b709976ca568d4"
server_key_version: "encryptionkey@0"
I20260925 09:27:06.741608 15692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "8d3c95b77ace4ee496b2745a4d6330d8"
format_stamp: "Formatted at 2026-09-25 09:27:06 on dist-test-slave-9pj8"
server_key: "30182f188322059a07d18a292157f02a"
server_key_iv: "ab3f32d38c077ac2a2b709976ca568d4"
server_key_version: "encryptionkey@0"
I20260925 09:27:06.746207 15692 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:27:06.749326 15706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:06.750538 15692 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 09:27:06.750697 15692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "8d3c95b77ace4ee496b2745a4d6330d8"
format_stamp: "Formatted at 2026-09-25 09:27:06 on dist-test-slave-9pj8"
server_key: "30182f188322059a07d18a292157f02a"
server_key_iv: "ab3f32d38c077ac2a2b709976ca568d4"
server_key_version: "encryptionkey@0"
I20260925 09:27:06.750893 15692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:06.756078 15692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:06.756737 15692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:06.757042 15692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:06.758212 15692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:06.759972 15692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:06.760079 15692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:06.760195 15692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:06.760269 15692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:06.792914 15692 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:38131
I20260925 09:27:06.793001 15818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:38131 every 8 connection(s)
I20260925 09:27:06.794166 15692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:27:06.802081 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:06.796735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:47181 (local address 127.10.114.126:39891)
0925 09:27:06.796967 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:06.796974 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:06.798377 (+ 1403us) server_negotiation.cc:407] Connection header received
0925 09:27:06.800013 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:06.800018 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:06.800071 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:06.800168 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:06.801369 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:06.801374 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:06.801377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:06.801394 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:06.801452 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:06.801465 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:06.801577 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:06.801783 (+ 206us) server_negotiation.cc:299] Negotiation successful
0925 09:27:06.801844 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260925 09:27:06.802163 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 15692
I20260925 09:27:06.802309 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:27:06.802825 10697 external_mini_cluster.cc:1428] Setting key 1a320532a9082fb02dfba0030b7dda00
I20260925 09:27:06.803225 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:06.797345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39891 (local address 127.10.114.67:47181)
0925 09:27:06.798216 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:06.798255 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:06.799688 (+ 1433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:06.800348 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:06.800356 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:06.800462 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:06.801100 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:27:06.801102 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:06.801137 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:27:06.801211 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:06.801558 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:06.801565 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:06.801570 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 09:27:06.801856 (+ 286us) client_negotiation.cc:240] Negotiation successful
0925 09:27:06.802096 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":262,"threads_started":1}
I20260925 09:27:06.804755 15819 heartbeater.cc:347] Connected to a master server at 127.10.114.126:39891
I20260925 09:27:06.805068 15819 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:06.805580 15819 heartbeater.cc:511] Master 127.10.114.126:39891 requested a full tablet report, sending...
I20260925 09:27:06.807005 15371 ts_manager.cc:194] Registered new tserver with Master: 8d3c95b77ace4ee496b2745a4d6330d8 (127.10.114.67:38131)
I20260925 09:27:06.818284 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:27:06.824971 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:06.822770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60138 (local address 127.10.114.126:39891)
0925 09:27:06.822918 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:06.822924 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:06.823387 (+ 463us) server_negotiation.cc:407] Connection header received
0925 09:27:06.823516 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:06.823519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:06.823564 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:06.823633 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:06.824507 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:06.824511 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:06.824514 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:06.824528 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:06.824555 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:06.824566 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:06.824652 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:06.824734 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 09:27:06.824775 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 09:27:06.828923 15371 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:27:06.835436 15371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:27:06.852499 15834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:06.847366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53832 (local address 127.10.114.65:42307)
0925 09:27:06.848036 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:06.848045 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:06.848648 (+ 603us) server_negotiation.cc:407] Connection header received
0925 09:27:06.848911 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:06.848920 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:06.849144 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:06.849262 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:06.851558 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:06.851568 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:06.851582 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:06.851610 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:06.851672 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:06.851694 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:06.851890 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:06.852065 (+ 175us) server_negotiation.cc:299] Negotiation successful
0925 09:27:06.852161 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":406,"threads_started":1}
I20260925 09:27:06.852686 15831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:06.847035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42307 (local address 127.0.0.1:53832)
0925 09:27:06.848503 (+ 1468us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:06.848548 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:06.848760 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:06.850261 (+ 1501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:06.850269 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:06.850352 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:06.851314 (+ 962us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:27:06.851319 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:06.851350 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:27:06.851417 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:06.851856 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:06.851865 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:06.851872 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 09:27:06.852077 (+ 205us) client_negotiation.cc:240] Negotiation successful
0925 09:27:06.852159 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1333,"spinlock_wait_cycles":10112,"thread_start_us":1014,"threads_started":1}
I20260925 09:27:06.852777 15833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:06.846817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:38131 (local address 127.0.0.1:57040)
0925 09:27:06.848501 (+ 1684us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:06.848548 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:06.848760 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:06.850261 (+ 1501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:06.850269 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:06.850352 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:06.851362 (+ 1010us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:27:06.851365 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:06.851375 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:27:06.851417 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:06.852030 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:06.852034 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:06.852039 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 09:27:06.852176 (+ 137us) client_negotiation.cc:240] Negotiation successful
0925 09:27:06.852226 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1400,"spinlock_wait_cycles":4608,"thread_start_us":1401,"threads_started":1}
I20260925 09:27:06.853490 15832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:06.847034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57040 (local address 127.10.114.67:38131)
0925 09:27:06.847612 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:06.847620 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:06.848646 (+ 1026us) server_negotiation.cc:407] Connection header received
0925 09:27:06.848911 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:06.848920 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:06.849144 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:06.849261 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:06.851672 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:06.851679 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:06.851692 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:06.851709 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:06.851767 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:06.851809 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:06.851976 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:06.852976 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0925 09:27:06.853080 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":468,"threads_started":1}
I20260925 09:27:06.854872 15835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:06.848871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:40983 (local address 127.0.0.1:46550)
0925 09:27:06.851165 (+ 2294us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:06.851183 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:06.851275 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:06.852948 (+ 1673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:06.852952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:06.852972 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:06.853638 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:27:06.853641 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:06.853655 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:27:06.853695 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:06.854186 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:06.854190 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:06.854196 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 09:27:06.854324 (+ 128us) client_negotiation.cc:240] Negotiation successful
0925 09:27:06.854370 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2182,"thread_start_us":2007,"threads_started":1}
I20260925 09:27:06.855463 15836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:06.849143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46550 (local address 127.10.114.66:40983)
0925 09:27:06.851538 (+ 2395us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:06.851557 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:06.851586 (+ 29us) server_negotiation.cc:407] Connection header received
0925 09:27:06.851651 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:06.851658 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:06.851899 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:06.852035 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:06.853859 (+ 1824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:06.853870 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:06.853883 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:06.853901 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:06.853955 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:06.853979 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:06.854131 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:06.855098 (+ 967us) server_negotiation.cc:299] Negotiation successful
0925 09:27:06.855200 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1864,"thread_start_us":2145,"threads_started":1}
I20260925 09:27:06.856771 15493 tablet_service.cc:1467] Processing CreateTablet for tablet 906f027809ae4104a25b6f0b2f84f4c8 (DEFAULT_TABLE table=test-table [id=27f5e46229814ea4b6c465edb9312e30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:06.856771 15754 tablet_service.cc:1467] Processing CreateTablet for tablet 906f027809ae4104a25b6f0b2f84f4c8 (DEFAULT_TABLE table=test-table [id=27f5e46229814ea4b6c465edb9312e30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:06.858145 15754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 906f027809ae4104a25b6f0b2f84f4c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:06.858146 15493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 906f027809ae4104a25b6f0b2f84f4c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:06.858490 15624 tablet_service.cc:1467] Processing CreateTablet for tablet 906f027809ae4104a25b6f0b2f84f4c8 (DEFAULT_TABLE table=test-table [id=27f5e46229814ea4b6c465edb9312e30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:06.859555 15624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 906f027809ae4104a25b6f0b2f84f4c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:06.869969 15837 tablet_bootstrap.cc:492] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916: Bootstrap starting.
I20260925 09:27:06.873050 15838 tablet_bootstrap.cc:492] T 906f027809ae4104a25b6f0b2f84f4c8 P 325484a91d9e414bac6d6fd1167da87d: Bootstrap starting.
I20260925 09:27:06.873533 15839 tablet_bootstrap.cc:492] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8: Bootstrap starting.
I20260925 09:27:06.873782 15837 tablet_bootstrap.cc:654] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:06.875038 15837 log.cc:824] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:06.876538 15838 tablet_bootstrap.cc:654] T 906f027809ae4104a25b6f0b2f84f4c8 P 325484a91d9e414bac6d6fd1167da87d: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:06.876832 15839 tablet_bootstrap.cc:654] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:06.877691 15838 log.cc:824] T 906f027809ae4104a25b6f0b2f84f4c8 P 325484a91d9e414bac6d6fd1167da87d: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:06.877728 15839 log.cc:824] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:06.878166 15837 tablet_bootstrap.cc:492] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916: No bootstrap required, opened a new log
I20260925 09:27:06.878525 15837 ts_tablet_manager.cc:1397] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260925 09:27:06.880618 15839 tablet_bootstrap.cc:492] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8: No bootstrap required, opened a new log
I20260925 09:27:06.880858 15839 ts_tablet_manager.cc:1397] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260925 09:27:06.885542 15838 tablet_bootstrap.cc:492] T 906f027809ae4104a25b6f0b2f84f4c8 P 325484a91d9e414bac6d6fd1167da87d: No bootstrap required, opened a new log
I20260925 09:27:06.885774 15838 ts_tablet_manager.cc:1397] T 906f027809ae4104a25b6f0b2f84f4c8 P 325484a91d9e414bac6d6fd1167da87d: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.003s
I20260925 09:27:06.886358 15837 raft_consensus.cc:348] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3c95b77ace4ee496b2745a4d6330d8" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38131 } } peers { permanent_uuid: "325484a91d9e414bac6d6fd1167da87d" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42307 } } peers { permanent_uuid: "b435ced4718b442298b7880433fb7916" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40983 } }
I20260925 09:27:06.886765 15837 raft_consensus.cc:374] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:06.886865 15837 raft_consensus.cc:729] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b435ced4718b442298b7880433fb7916, State: Initialized, Role: FOLLOWER
I20260925 09:27:06.887465 15837 consensus_queue.cc:260] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3c95b77ace4ee496b2745a4d6330d8" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38131 } } peers { permanent_uuid: "325484a91d9e414bac6d6fd1167da87d" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42307 } } peers { permanent_uuid: "b435ced4718b442298b7880433fb7916" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40983 } }
I20260925 09:27:06.887606 15839 raft_consensus.cc:348] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3c95b77ace4ee496b2745a4d6330d8" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38131 } } peers { permanent_uuid: "325484a91d9e414bac6d6fd1167da87d" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42307 } } peers { permanent_uuid: "b435ced4718b442298b7880433fb7916" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40983 } }
I20260925 09:27:06.888362 15839 raft_consensus.cc:374] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:06.888515 15839 raft_consensus.cc:729] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d3c95b77ace4ee496b2745a4d6330d8, State: Initialized, Role: FOLLOWER
I20260925 09:27:06.889098 15839 consensus_queue.cc:260] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3c95b77ace4ee496b2745a4d6330d8" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38131 } } peers { permanent_uuid: "325484a91d9e414bac6d6fd1167da87d" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42307 } } peers { permanent_uuid: "b435ced4718b442298b7880433fb7916" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40983 } }
I20260925 09:27:06.890129 15819 heartbeater.cc:503] Master 127.10.114.126:39891 was elected leader, sending a full tablet report...
I20260925 09:27:06.890700 15839 ts_tablet_manager.cc:1428] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8: Time spent starting tablet: real 0.010s user 0.006s sys 0.003s
I20260925 09:27:06.891351 15689 heartbeater.cc:503] Master 127.10.114.126:39891 was elected leader, sending a full tablet report...
I20260925 09:27:06.892287 15837 ts_tablet_manager.cc:1428] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916: Time spent starting tablet: real 0.014s user 0.005s sys 0.006s
I20260925 09:27:06.895582 15838 raft_consensus.cc:348] T 906f027809ae4104a25b6f0b2f84f4c8 P 325484a91d9e414bac6d6fd1167da87d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3c95b77ace4ee496b2745a4d6330d8" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38131 } } peers { permanent_uuid: "325484a91d9e414bac6d6fd1167da87d" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42307 } } peers { permanent_uuid: "b435ced4718b442298b7880433fb7916" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40983 } }
I20260925 09:27:06.895965 15838 raft_consensus.cc:374] T 906f027809ae4104a25b6f0b2f84f4c8 P 325484a91d9e414bac6d6fd1167da87d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:06.896070 15838 raft_consensus.cc:729] T 906f027809ae4104a25b6f0b2f84f4c8 P 325484a91d9e414bac6d6fd1167da87d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 325484a91d9e414bac6d6fd1167da87d, State: Initialized, Role: FOLLOWER
I20260925 09:27:06.896624 15838 consensus_queue.cc:260] T 906f027809ae4104a25b6f0b2f84f4c8 P 325484a91d9e414bac6d6fd1167da87d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3c95b77ace4ee496b2745a4d6330d8" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38131 } } peers { permanent_uuid: "325484a91d9e414bac6d6fd1167da87d" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42307 } } peers { permanent_uuid: "b435ced4718b442298b7880433fb7916" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40983 } }
I20260925 09:27:06.897604 15558 heartbeater.cc:503] Master 127.10.114.126:39891 was elected leader, sending a full tablet report...
I20260925 09:27:06.898088 15838 ts_tablet_manager.cc:1428] T 906f027809ae4104a25b6f0b2f84f4c8 P 325484a91d9e414bac6d6fd1167da87d: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
I20260925 09:27:06.927968 15843 raft_consensus.cc:482] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:27:06.928236 15843 raft_consensus.cc:504] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3c95b77ace4ee496b2745a4d6330d8" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38131 } } peers { permanent_uuid: "325484a91d9e414bac6d6fd1167da87d" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42307 } } peers { permanent_uuid: "b435ced4718b442298b7880433fb7916" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40983 } }
I20260925 09:27:06.929749 15843 leader_election.cc:288] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d3c95b77ace4ee496b2745a4d6330d8 (127.10.114.67:38131), 325484a91d9e414bac6d6fd1167da87d (127.10.114.65:42307)
I20260925 09:27:06.933383 15832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:06.930145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:37543 (local address 127.10.114.67:38131)
0925 09:27:06.930363 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:06.930373 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:06.931066 (+ 693us) server_negotiation.cc:407] Connection header received
0925 09:27:06.931275 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:06.931281 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:06.931346 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:06.931441 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:06.932641 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:06.932647 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:06.932651 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:06.932670 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:06.932709 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:06.932720 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:06.932806 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:06.933052 (+ 246us) server_negotiation.cc:299] Negotiation successful
0925 09:27:06.933112 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 09:27:06.933454 15847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:06.930047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:38131 (local address 127.10.114.66:37543)
0925 09:27:06.930950 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:06.930971 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:06.931127 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:06.931487 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:06.931491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:06.931517 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:06.932427 (+ 910us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:27:06.932431 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:06.932448 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:27:06.932500 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:06.932820 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:06.932826 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:06.932833 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 09:27:06.932933 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 09:27:06.932979 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":502,"threads_started":1}
I20260925 09:27:06.933490 15834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:06.930292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:49453 (local address 127.10.114.65:42307)
0925 09:27:06.930548 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:06.930557 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:06.931266 (+ 709us) server_negotiation.cc:407] Connection header received
0925 09:27:06.931392 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:06.931396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:06.931478 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:06.931576 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:06.932703 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:06.932707 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:06.932710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:06.932724 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:06.932750 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:06.932784 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:06.932897 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:06.933151 (+ 254us) server_negotiation.cc:299] Negotiation successful
0925 09:27:06.933202 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260925 09:27:06.933760 15846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:06.930077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42307 (local address 127.10.114.66:49453)
0925 09:27:06.931127 (+ 1050us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:06.931145 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:06.931272 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:06.931630 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:06.931633 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:06.931654 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:06.932499 (+ 845us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:27:06.932501 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:06.932512 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:27:06.932566 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:06.933011 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:06.933020 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:06.933025 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 09:27:06.933163 (+ 138us) client_negotiation.cc:240] Negotiation successful
0925 09:27:06.933226 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"thread_start_us":707,"threads_started":1}
I20260925 09:27:06.934582 15774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "906f027809ae4104a25b6f0b2f84f4c8" candidate_uuid: "b435ced4718b442298b7880433fb7916" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d3c95b77ace4ee496b2745a4d6330d8" is_pre_election: true
I20260925 09:27:06.934682 15513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "906f027809ae4104a25b6f0b2f84f4c8" candidate_uuid: "b435ced4718b442298b7880433fb7916" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "325484a91d9e414bac6d6fd1167da87d" is_pre_election: true
I20260925 09:27:06.935065 15774 raft_consensus.cc:2454] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b435ced4718b442298b7880433fb7916 in term 0.
I20260925 09:27:06.935168 15513 raft_consensus.cc:2454] T 906f027809ae4104a25b6f0b2f84f4c8 P 325484a91d9e414bac6d6fd1167da87d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b435ced4718b442298b7880433fb7916 in term 0.
I20260925 09:27:06.935755 15577 leader_election.cc:302] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d3c95b77ace4ee496b2745a4d6330d8, b435ced4718b442298b7880433fb7916; no voters:
I20260925 09:27:06.936151 15843 raft_consensus.cc:2781] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:27:06.936296 15843 raft_consensus.cc:482] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:27:06.936416 15843 raft_consensus.cc:3037] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:06.938705 15843 raft_consensus.cc:504] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3c95b77ace4ee496b2745a4d6330d8" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38131 } } peers { permanent_uuid: "325484a91d9e414bac6d6fd1167da87d" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42307 } } peers { permanent_uuid: "b435ced4718b442298b7880433fb7916" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40983 } }
I20260925 09:27:06.939248 15843 leader_election.cc:288] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [CANDIDATE]: Term 1 election: Requested vote from peers 8d3c95b77ace4ee496b2745a4d6330d8 (127.10.114.67:38131), 325484a91d9e414bac6d6fd1167da87d (127.10.114.65:42307)
I20260925 09:27:06.939723 15774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "906f027809ae4104a25b6f0b2f84f4c8" candidate_uuid: "b435ced4718b442298b7880433fb7916" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d3c95b77ace4ee496b2745a4d6330d8"
I20260925 09:27:06.939723 15513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "906f027809ae4104a25b6f0b2f84f4c8" candidate_uuid: "b435ced4718b442298b7880433fb7916" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "325484a91d9e414bac6d6fd1167da87d"
I20260925 09:27:06.939931 15774 raft_consensus.cc:3037] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:06.939930 15513 raft_consensus.cc:3037] T 906f027809ae4104a25b6f0b2f84f4c8 P 325484a91d9e414bac6d6fd1167da87d [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:06.943063 15774 raft_consensus.cc:2454] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b435ced4718b442298b7880433fb7916 in term 1.
I20260925 09:27:06.943068 15513 raft_consensus.cc:2454] T 906f027809ae4104a25b6f0b2f84f4c8 P 325484a91d9e414bac6d6fd1167da87d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b435ced4718b442298b7880433fb7916 in term 1.
I20260925 09:27:06.943418 15580 leader_election.cc:302] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 325484a91d9e414bac6d6fd1167da87d, b435ced4718b442298b7880433fb7916; no voters:
I20260925 09:27:06.943730 15843 raft_consensus.cc:2781] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:06.944157 15843 raft_consensus.cc:686] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [term 1 LEADER]: Becoming Leader. State: Replica: b435ced4718b442298b7880433fb7916, State: Running, Role: LEADER
I20260925 09:27:06.944588 15843 consensus_queue.cc:237] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3c95b77ace4ee496b2745a4d6330d8" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38131 } } peers { permanent_uuid: "325484a91d9e414bac6d6fd1167da87d" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42307 } } peers { permanent_uuid: "b435ced4718b442298b7880433fb7916" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40983 } }
I20260925 09:27:06.949283 15372 catalog_manager.cc:5495] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 reported cstate change: term changed from 0 to 1, leader changed from <none> to b435ced4718b442298b7880433fb7916 (127.10.114.66). New cstate: current_term: 1 leader_uuid: "b435ced4718b442298b7880433fb7916" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3c95b77ace4ee496b2745a4d6330d8" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "325484a91d9e414bac6d6fd1167da87d" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b435ced4718b442298b7880433fb7916" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 40983 } health_report { overall_health: HEALTHY } } }
W20260925 09:27:06.966145 15559 tablet.cc:2369] T 906f027809ae4104a25b6f0b2f84f4c8 P 325484a91d9e414bac6d6fd1167da87d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:27:06.978989 15836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:06.976948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46562 (local address 127.10.114.66:40983)
0925 09:27:06.977129 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:06.977134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:06.977154 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:27:06.977298 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:06.977301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:06.977369 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:06.977498 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:06.978382 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:06.978387 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:06.978392 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:06.978408 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:06.978447 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:06.978459 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:06.978583 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:06.978706 (+ 123us) server_negotiation.cc:299] Negotiation successful
0925 09:27:06.978755 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:27:06.986693 15513 raft_consensus.cc:1261] T 906f027809ae4104a25b6f0b2f84f4c8 P 325484a91d9e414bac6d6fd1167da87d [term 1 FOLLOWER]: Refusing update from remote peer b435ced4718b442298b7880433fb7916: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:27:06.986694 15774 raft_consensus.cc:1261] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8 [term 1 FOLLOWER]: Refusing update from remote peer b435ced4718b442298b7880433fb7916: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:27:06.987603 15843 consensus_queue.cc:1035] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [LEADER]: Connected to new peer: Peer: permanent_uuid: "325484a91d9e414bac6d6fd1167da87d" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:27:06.987910 15848 consensus_queue.cc:1035] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d3c95b77ace4ee496b2745a4d6330d8" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:27:06.998018 15851 mvcc.cc:205] Tried to move back new op lower bound from 7333185236927270912 to 7333185236768575488. Current Snapshot: MvccSnapshot[applied={T|T < 7333185236927270912}]
I20260925 09:27:07.013520 15371 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:27:07.013768 15371 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:27:07.017640 15371 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 12d0c8f2d5f44a8aa4f23cff11604c5f: Sending DeleteTablet for 3 replicas of tablet 906f027809ae4104a25b6f0b2f84f4c8
I20260925 09:27:07.018743 15754 tablet_service.cc:1514] Processing DeleteTablet for tablet 906f027809ae4104a25b6f0b2f84f4c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:07 UTC) from {username='slave'} at 127.0.0.1:57040
I20260925 09:27:07.018862 15624 tablet_service.cc:1514] Processing DeleteTablet for tablet 906f027809ae4104a25b6f0b2f84f4c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:07 UTC) from {username='slave'} at 127.0.0.1:46550
I20260925 09:27:07.019057 15493 tablet_service.cc:1514] Processing DeleteTablet for tablet 906f027809ae4104a25b6f0b2f84f4c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:07 UTC) from {username='slave'} at 127.0.0.1:53832
I20260925 09:27:07.021010 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 15431
I20260925 09:27:07.028595 15863 tablet_replica.cc:330] T 906f027809ae4104a25b6f0b2f84f4c8 P 325484a91d9e414bac6d6fd1167da87d: stopping tablet replica
I20260925 09:27:07.029132 15863 raft_consensus.cc:2229] T 906f027809ae4104a25b6f0b2f84f4c8 P 325484a91d9e414bac6d6fd1167da87d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:27:07.029753 15863 raft_consensus.cc:2258] T 906f027809ae4104a25b6f0b2f84f4c8 P 325484a91d9e414bac6d6fd1167da87d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:07.032053 15863 ts_tablet_manager.cc:1905] T 906f027809ae4104a25b6f0b2f84f4c8 P 325484a91d9e414bac6d6fd1167da87d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:07.032636 15861 tablet_replica.cc:330] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8: stopping tablet replica
I20260925 09:27:07.032891 15862 tablet_replica.cc:330] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916: stopping tablet replica
I20260925 09:27:07.033010 15861 raft_consensus.cc:2229] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:27:07.033291 15862 raft_consensus.cc:2229] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:27:07.033452 15861 raft_consensus.cc:2258] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:07.032994 15834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:07.022411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53842 (local address 127.10.114.65:42307)
0925 09:27:07.022641 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:07.022650 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:07.026810 (+ 4160us) server_negotiation.cc:407] Connection header received
0925 09:27:07.026938 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:07.026947 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:07.027081 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:07.027203 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:07.032233 (+ 5030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:07.032241 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:07.032248 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:07.032269 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:07.032310 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:07.032329 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:07.032424 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:07.032673 (+ 249us) server_negotiation.cc:299] Negotiation successful
0925 09:27:07.032733 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 09:27:07.034300 15862 raft_consensus.cc:2258] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:07.034358 15553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:07.036448 15862 ts_tablet_manager.cc:1905] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:07.040937 15861 ts_tablet_manager.cc:1905] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:07.043491 15862 ts_tablet_manager.cc:1918] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:27:07.043643 15862 log.cc:1196] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/906f027809ae4104a25b6f0b2f84f4c8
I20260925 09:27:07.044211 15862 ts_tablet_manager.cc:1939] T 906f027809ae4104a25b6f0b2f84f4c8 P b435ced4718b442298b7880433fb7916: Deleting consensus metadata
I20260925 09:27:07.045886 15359 catalog_manager.cc:4841] TS b435ced4718b442298b7880433fb7916 (127.10.114.66:40983): tablet 906f027809ae4104a25b6f0b2f84f4c8 (table test-table [id=27f5e46229814ea4b6c465edb9312e30]) successfully deleted
I20260925 09:27:07.048563 15861 ts_tablet_manager.cc:1918] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:27:07.048743 15861 log.cc:1196] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/906f027809ae4104a25b6f0b2f84f4c8
I20260925 09:27:07.049300 15861 ts_tablet_manager.cc:1939] T 906f027809ae4104a25b6f0b2f84f4c8 P 8d3c95b77ace4ee496b2745a4d6330d8: Deleting consensus metadata
I20260925 09:27:07.050976 15356 catalog_manager.cc:4841] TS 8d3c95b77ace4ee496b2745a4d6330d8 (127.10.114.67:38131): tablet 906f027809ae4104a25b6f0b2f84f4c8 (table test-table [id=27f5e46229814ea4b6c465edb9312e30]) successfully deleted
I20260925 09:27:07.123389 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 15431
I20260925 09:27:07.141256 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 15562
W20260925 09:27:07.141197 15359 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.114.65:42307 (error 108)
I20260925 09:27:07.143347 15835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:07.142800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42307 (local address 127.0.0.1:53852)
0925 09:27:07.143069 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:07.143192 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
W20260925 09:27:07.143594 15359 catalog_manager.cc:4584] TS 325484a91d9e414bac6d6fd1167da87d (127.10.114.65:42307): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 906f027809ae4104a25b6f0b2f84f4c8: Network error: Client connection negotiation failed: client connection to 127.10.114.65:42307: connect: Connection refused (error 111)
I20260925 09:27:07.144474 15836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:07.142370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46572 (local address 127.10.114.66:40983)
0925 09:27:07.142530 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:07.142535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:07.142550 (+ 15us) server_negotiation.cc:407] Connection header received
0925 09:27:07.142628 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:07.142632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:07.142677 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:07.142808 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:07.143812 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:07.143822 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:07.143826 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:07.143844 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:07.143878 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:07.143893 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:07.144049 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:07.144141 (+ 92us) server_negotiation.cc:299] Negotiation successful
0925 09:27:07.144211 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 09:27:07.145213 15684 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:07.197090 15835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:07.196608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42307 (local address 127.0.0.1:53856)
0925 09:27:07.196805 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:07.196914 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260925 09:27:07.198137 15835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:07.197770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42307 (local address 127.0.0.1:53864)
0925 09:27:07.197920 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:07.197999 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260925 09:27:07.249716 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 15562
I20260925 09:27:07.258860 15835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:07.258318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42307 (local address 127.0.0.1:53880)
0925 09:27:07.258551 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:07.258685 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260925 09:27:07.260036 15835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:07.259651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42307 (local address 127.0.0.1:53892)
0925 09:27:07.259848 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:07.259922 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260925 09:27:07.268121 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 15692
I20260925 09:27:07.270630 15832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:07.268833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57054 (local address 127.10.114.67:38131)
0925 09:27:07.269014 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:07.269018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:07.269035 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:27:07.269143 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:07.269147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:07.269212 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:07.269290 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:07.270058 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:07.270062 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:07.270065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:07.270078 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:07.270106 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:07.270117 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:07.270215 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:07.270290 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 09:27:07.270329 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:27:07.271209 15814 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:07.340271 15835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:07.339821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42307 (local address 127.0.0.1:53904)
0925 09:27:07.340015 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:07.340113 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260925 09:27:07.341296 15835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:07.340908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:42307 (local address 127.0.0.1:53918)
0925 09:27:07.341107 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:07.341172 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260925 09:27:07.356431 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 15692
I20260925 09:27:07.372802 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 15341
I20260925 09:27:07.373898 15402 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:07.475217 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 15341
2026-09-25T09:27:07Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2477 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260925 09:27:07.515214 10697 mini_oidc.cc:152] Starting JWKS server
I20260925 09:27:07.517729 10697 webserver.cc:460] Webserver started at https://127.0.0.1:40783/ using document root <none> and password file <none>
I20260925 09:27:07.518070 10697 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 09:27:07.518710 10697 webserver.cc:460] Webserver started at http://127.0.0.1:35911/ using document root <none> and password file <none>
2026-09-25T09:27:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:27:07Z Disabled control of system clock
I20260925 09:27:07.546406 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:44227
--webserver_interface=127.10.114.126
--webserver_port=0
--builtin_ntp_servers=127.10.114.84:45211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:44227
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40783/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790328385296020-10697-0/testchainca.cert with env {}
W20260925 09:27:07.896191 15875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:07.896492 15875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:07.896593 15875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:07.904529 15875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:27:07.904641 15875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:07.904688 15875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:07.904757 15875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:27:07.904814 15875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 09:27:07.904886 15875 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 09:27:07.904939 15875 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40783/jwks.json
I20260925 09:27:07.915570 15875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:45211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:44227
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:44227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40783/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790328385296020-10697-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:07.916925 15875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:07.918243 15875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:07.930987 15880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:07.931108 15881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:07.931128 15883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:07.932451 15875 server_base.cc:1034] running on GCE node
I20260925 09:27:07.933184 15875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:07.934512 15875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:07.935737 15875 hybrid_clock.cc:648] HybridClock initialized: now 1790328427935689 us; error 45 us; skew 500 ppm
I20260925 09:27:07.936103 15875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:27:07.938225 15875 webserver.cc:460] Webserver started at http://127.10.114.126:37343/ using document root <none> and password file <none>
I20260925 09:27:07.938870 15875 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:07.939004 15875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:07.939313 15875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:07.942050 15875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "0a5c90da33ef4ab883331ef7433bbc5e"
format_stamp: "Formatted at 2026-09-25 09:27:07 on dist-test-slave-9pj8"
server_key: "632b6f786fcc6e0332f09129604c8e28"
server_key_iv: "688744a564f68276d796f79f9eb3bb45"
server_key_version: "encryptionkey@0"
I20260925 09:27:07.942801 15875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "0a5c90da33ef4ab883331ef7433bbc5e"
format_stamp: "Formatted at 2026-09-25 09:27:07 on dist-test-slave-9pj8"
server_key: "632b6f786fcc6e0332f09129604c8e28"
server_key_iv: "688744a564f68276d796f79f9eb3bb45"
server_key_version: "encryptionkey@0"
I20260925 09:27:07.947459 15875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260925 09:27:07.950719 15889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:07.952104 15875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:27:07.952276 15875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "0a5c90da33ef4ab883331ef7433bbc5e"
format_stamp: "Formatted at 2026-09-25 09:27:07 on dist-test-slave-9pj8"
server_key: "632b6f786fcc6e0332f09129604c8e28"
server_key_iv: "688744a564f68276d796f79f9eb3bb45"
server_key_version: "encryptionkey@0"
I20260925 09:27:07.952502 15875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:07.969907 15875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:07.970670 15875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:07.970971 15875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:07.989109 15875 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:44227
I20260925 09:27:07.989131 15940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:44227 every 8 connection(s)
I20260925 09:27:07.990356 15875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:27:07.993662 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 15875
I20260925 09:27:07.993875 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:27:07.994105 15941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:07.994549 10697 external_mini_cluster.cc:1428] Setting key 4901455245e6442918dabb034a66a402
I20260925 09:27:08.009243 15944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:07.996398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51150 (local address 127.10.114.126:44227)
0925 09:27:07.997508 (+ 1110us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:07.997527 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:07.997579 (+ 52us) server_negotiation.cc:407] Connection header received
0925 09:27:07.999286 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:07.999320 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:07.999925 (+ 605us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:08.000349 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:08.000887 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:08.002853 (+ 1966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:08.004201 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:08.005222 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:08.005412 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:08.006504 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:08.006515 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:08.006541 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:08.007280 (+ 739us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:08.007374 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:08.007398 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:08.007505 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:08.007693 (+ 188us) server_negotiation.cc:299] Negotiation successful
0925 09:27:08.008034 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":350,"threads_started":1}
I20260925 09:27:08.009333 15941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e: Bootstrap starting.
I20260925 09:27:08.012583 15941 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:08.013095 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:27:08.013803 15941 log.cc:824] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:08.016690 15941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e: No bootstrap required, opened a new log
I20260925 09:27:08.023660 15941 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5c90da33ef4ab883331ef7433bbc5e" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44227 } }
I20260925 09:27:08.024106 15941 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:08.024245 15941 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a5c90da33ef4ab883331ef7433bbc5e, State: Initialized, Role: FOLLOWER
I20260925 09:27:08.024881 15941 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5c90da33ef4ab883331ef7433bbc5e" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44227 } }
I20260925 09:27:08.025103 15941 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:27:08.025225 15941 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:27:08.025410 15941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:08.027478 15941 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5c90da33ef4ab883331ef7433bbc5e" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44227 } }
I20260925 09:27:08.028035 15941 leader_election.cc:302] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a5c90da33ef4ab883331ef7433bbc5e; no voters:
I20260925 09:27:08.028720 15941 leader_election.cc:288] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:27:08.028867 15946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:08.029852 15946 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e [term 1 LEADER]: Becoming Leader. State: Replica: 0a5c90da33ef4ab883331ef7433bbc5e, State: Running, Role: LEADER
I20260925 09:27:08.030385 15946 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5c90da33ef4ab883331ef7433bbc5e" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44227 } }
I20260925 09:27:08.030910 15941 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:27:08.033246 15947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a5c90da33ef4ab883331ef7433bbc5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5c90da33ef4ab883331ef7433bbc5e" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44227 } } }
I20260925 09:27:08.033684 15947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:08.034150 15948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a5c90da33ef4ab883331ef7433bbc5e. Latest consensus state: current_term: 1 leader_uuid: "0a5c90da33ef4ab883331ef7433bbc5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5c90da33ef4ab883331ef7433bbc5e" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44227 } } }
I20260925 09:27:08.034428 15948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:08.035480 15954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:27:08.040162 15954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:27:08.047959 15954 catalog_manager.cc:1347] Generated new cluster ID: 360d255bcdbc478985c90e39232d3a6f
I20260925 09:27:08.048051 15954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:27:08.060390 15954 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:27:08.061481 15954 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:27:08.067183 15954 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a5c90da33ef4ab883331ef7433bbc5e: Generated new TSK 0
I20260925 09:27:08.067912 15954 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 09:27:08.106144 15971 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260925 09:27:08.106472 15971 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260925 09:27:08.108942 15970 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:08.085130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:44227 (local address 127.0.0.1:51158)
0925 09:27:08.085948 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:08.085980 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:08.086147 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:08.086685 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:08.086689 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:08.086700 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0925 09:27:08.087055 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:08.087064 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:08.088502 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:08.088516 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:08.089623 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:08.089636 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:08.090635 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:08.090640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:08.090761 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:08.090818 (+ 57us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 09:27:08.106841 (+ 16023us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40783/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 09:27:08.107158 (+ 317us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.114.126:44227: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40783/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":461,"threads_started":1}
I20260925 09:27:08.109998 15944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:08.085256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51158 (local address 127.10.114.126:44227)
0925 09:27:08.085447 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:08.085453 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:08.086092 (+ 639us) server_negotiation.cc:407] Connection header received
0925 09:27:08.086374 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:08.086382 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:08.086470 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:08.086650 (+ 180us) server_negotiation.cc:226] Negotiated authn=JWT
0925 09:27:08.087219 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:08.088323 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:08.089772 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:08.090462 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:08.090592 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:08.091107 (+ 515us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 09:27:08.106536 (+ 15429us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40783/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 09:27:08.109696 (+ 3160us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51158: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40783/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":65}
W20260925 09:27:08.110121 15944 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51158: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40783/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.10.114.126:44227: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40783/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260925 09:27:08.111442 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 15875
I20260925 09:27:08.112694 15936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:08.201048 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 15875
2026-09-25T09:27:08Z chronyd exiting
I20260925 09:27:08.251049 10697 test_util.cc:175] -----------------------------------------------
I20260925 09:27:08.251199 10697 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790328385296020-10697-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (740 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260925 09:27:08.255422 10697 mini_oidc.cc:152] Starting JWKS server
I20260925 09:27:08.257210 10697 webserver.cc:460] Webserver started at https://127.0.0.1:40373/ using document root <none> and password file <none>
I20260925 09:27:08.257490 10697 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 09:27:08.258132 10697 webserver.cc:460] Webserver started at http://127.0.0.1:43143/ using document root <none> and password file <none>
2026-09-25T09:27:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:27:08Z Disabled control of system clock
I20260925 09:27:08.283524 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:39089
--webserver_interface=127.10.114.126
--webserver_port=0
--builtin_ntp_servers=127.10.114.84:40425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:39089
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40373/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790328385296020-10697-0/testchainca.cert with env {}
W20260925 09:27:08.604878 15978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:08.605167 15978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:08.605260 15978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:08.612586 15978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:27:08.612689 15978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:08.612731 15978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:08.612784 15978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:27:08.612834 15978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 09:27:08.612895 15978 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 09:27:08.612938 15978 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40373/jwks.json
I20260925 09:27:08.623049 15978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:40425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:39089
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:39089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40373/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790328385296020-10697-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:08.624297 15978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:08.625623 15978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:08.637581 15986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:08.637668 15983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:08.637640 15984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:08.643676 15978 server_base.cc:1034] running on GCE node
I20260925 09:27:08.644488 15978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:08.645840 15978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:08.647101 15978 hybrid_clock.cc:648] HybridClock initialized: now 1790328428647062 us; error 52 us; skew 500 ppm
I20260925 09:27:08.647468 15978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:27:08.650087 15978 webserver.cc:460] Webserver started at http://127.10.114.126:33313/ using document root <none> and password file <none>
I20260925 09:27:08.650763 15978 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:08.650892 15978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:08.651232 15978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:08.654052 15978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "fa065e742ec94cce9453eb9269960947"
format_stamp: "Formatted at 2026-09-25 09:27:08 on dist-test-slave-9pj8"
server_key: "e1cd41bd1ef603bf5b84ac20b9ab3475"
server_key_iv: "6b4cfcbaa84d803428d5a2d1ca53e3d3"
server_key_version: "encryptionkey@0"
I20260925 09:27:08.654831 15978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "fa065e742ec94cce9453eb9269960947"
format_stamp: "Formatted at 2026-09-25 09:27:08 on dist-test-slave-9pj8"
server_key: "e1cd41bd1ef603bf5b84ac20b9ab3475"
server_key_iv: "6b4cfcbaa84d803428d5a2d1ca53e3d3"
server_key_version: "encryptionkey@0"
I20260925 09:27:08.659729 15978 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260925 09:27:08.662948 15992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:08.664280 15978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:27:08.664470 15978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "fa065e742ec94cce9453eb9269960947"
format_stamp: "Formatted at 2026-09-25 09:27:08 on dist-test-slave-9pj8"
server_key: "e1cd41bd1ef603bf5b84ac20b9ab3475"
server_key_iv: "6b4cfcbaa84d803428d5a2d1ca53e3d3"
server_key_version: "encryptionkey@0"
I20260925 09:27:08.664669 15978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:08.677937 15978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:08.678643 15978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:08.678972 15978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:08.696645 15978 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:39089
I20260925 09:27:08.696659 16043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:39089 every 8 connection(s)
I20260925 09:27:08.697907 15978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:27:08.699611 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 15978
I20260925 09:27:08.699817 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:27:08.700453 10697 external_mini_cluster.cc:1428] Setting key cbe76b9734dc299571ae860a93811e5f
I20260925 09:27:08.701972 16044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:08.714915 16047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:08.702279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42888 (local address 127.10.114.126:39089)
0925 09:27:08.703304 (+ 1025us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:08.703320 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:08.703371 (+ 51us) server_negotiation.cc:407] Connection header received
0925 09:27:08.705174 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:08.705199 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:08.705793 (+ 594us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:08.706186 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:08.706675 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:08.708751 (+ 2076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:08.709966 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:08.710925 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:08.711126 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:08.712254 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:08.712264 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:08.712280 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:08.712972 (+ 692us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:08.713067 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:08.713097 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:08.713218 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:08.713438 (+ 220us) server_negotiation.cc:299] Negotiation successful
0925 09:27:08.713742 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":322,"threads_started":1}
I20260925 09:27:08.715885 16044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947: Bootstrap starting.
I20260925 09:27:08.718469 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:27:08.719020 16044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:08.720247 16044 log.cc:824] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:08.723383 16044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947: No bootstrap required, opened a new log
I20260925 09:27:08.731521 16044 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa065e742ec94cce9453eb9269960947" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39089 } }
I20260925 09:27:08.731910 16044 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:08.732026 16044 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa065e742ec94cce9453eb9269960947, State: Initialized, Role: FOLLOWER
I20260925 09:27:08.732659 16044 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa065e742ec94cce9453eb9269960947" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39089 } }
I20260925 09:27:08.732909 16044 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:27:08.733064 16044 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:27:08.733258 16044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:08.735908 16044 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa065e742ec94cce9453eb9269960947" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39089 } }
I20260925 09:27:08.736443 16044 leader_election.cc:302] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa065e742ec94cce9453eb9269960947; no voters:
I20260925 09:27:08.737046 16044 leader_election.cc:288] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:27:08.737183 16049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:08.737886 16049 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947 [term 1 LEADER]: Becoming Leader. State: Replica: fa065e742ec94cce9453eb9269960947, State: Running, Role: LEADER
I20260925 09:27:08.738396 16049 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa065e742ec94cce9453eb9269960947" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39089 } }
I20260925 09:27:08.739192 16044 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:27:08.741351 16051 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa065e742ec94cce9453eb9269960947. Latest consensus state: current_term: 1 leader_uuid: "fa065e742ec94cce9453eb9269960947" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa065e742ec94cce9453eb9269960947" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39089 } } }
I20260925 09:27:08.741178 16050 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa065e742ec94cce9453eb9269960947" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa065e742ec94cce9453eb9269960947" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39089 } } }
I20260925 09:27:08.741650 16051 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:08.741734 16050 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:08.744660 16056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:27:08.749737 16056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:27:08.757995 16056 catalog_manager.cc:1347] Generated new cluster ID: 59bd7169ea3f40a0ace1727a40fc7e7a
I20260925 09:27:08.758117 16056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:27:08.770728 16056 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:27:08.772048 16056 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:27:08.786620 16056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa065e742ec94cce9453eb9269960947: Generated new TSK 0
I20260925 09:27:08.787427 16056 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 09:27:08.829396 16074 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260925 09:27:08.829520 16074 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 09:27:08.830798 16047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:08.811130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42890 (local address 127.10.114.126:39089)
0925 09:27:08.811330 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:08.811337 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:08.811753 (+ 416us) server_negotiation.cc:407] Connection header received
0925 09:27:08.812044 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:08.812050 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:08.812144 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:08.812314 (+ 170us) server_negotiation.cc:226] Negotiated authn=JWT
0925 09:27:08.812749 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:08.813843 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:08.815155 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:08.815924 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:08.816058 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:08.816485 (+ 427us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 09:27:08.830054 (+ 13569us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40373/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 09:27:08.830455 (+ 401us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42890: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40373/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":63}
W20260925 09:27:08.830873 16073 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:08.810967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39089 (local address 127.0.0.1:42890)
0925 09:27:08.811651 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:08.811666 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:08.811839 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:08.812325 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:08.812328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:08.812337 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0925 09:27:08.812594 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:08.812601 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:08.813995 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:08.813999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:08.815011 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:08.815022 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:08.816069 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:08.816073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:08.816197 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:08.816226 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 09:27:08.830278 (+ 14052us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40373/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 09:27:08.830614 (+ 336us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.114.126:39089: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40373/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":410,"threads_started":1}
W20260925 09:27:08.830945 16047 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42890: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40373/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260925 09:27:08.832666 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 15978
I20260925 09:27:08.833914 16039 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:08.923944 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 15978
2026-09-25T09:27:08Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (723 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260925 09:27:08.978734 10697 mini_oidc.cc:152] Starting JWKS server
I20260925 09:27:08.980629 10697 webserver.cc:460] Webserver started at https://127.0.0.1:46689/ using document root <none> and password file <none>
I20260925 09:27:08.980870 10697 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 09:27:08.981479 10697 webserver.cc:460] Webserver started at http://127.0.0.1:46331/ using document root <none> and password file <none>
2026-09-25T09:27:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:27:08Z Disabled control of system clock
I20260925 09:27:09.006747 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:33693
--webserver_interface=127.10.114.126
--webserver_port=0
--builtin_ntp_servers=127.10.114.84:45829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:33693
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46689/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 09:27:09.313643 16081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:09.313889 16081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:09.314064 16081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:09.321074 16081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:27:09.321230 16081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:09.321336 16081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:09.321470 16081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:27:09.321544 16081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 09:27:09.321641 16081 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 09:27:09.321704 16081 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46689/jwks.json
I20260925 09:27:09.331493 16081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:45829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:33693
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:33693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46689/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:09.332823 16081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:09.334659 16081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:09.346536 16086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:09.346474 16087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:09.346575 16089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:09.347404 16081 server_base.cc:1034] running on GCE node
I20260925 09:27:09.348171 16081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:09.349447 16081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:09.350706 16081 hybrid_clock.cc:648] HybridClock initialized: now 1790328429350670 us; error 47 us; skew 500 ppm
I20260925 09:27:09.351102 16081 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:27:09.357195 16081 webserver.cc:460] Webserver started at http://127.10.114.126:33861/ using document root <none> and password file <none>
I20260925 09:27:09.357887 16081 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:09.358019 16081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:09.358355 16081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:09.361177 16081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "ed4f805faef848f38cd2536d9b18faa9"
format_stamp: "Formatted at 2026-09-25 09:27:09 on dist-test-slave-9pj8"
server_key: "f01985856eb103150f7b743c216ffd02"
server_key_iv: "21be09f2d5035919b77cda38d511dbaf"
server_key_version: "encryptionkey@0"
I20260925 09:27:09.361966 16081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "ed4f805faef848f38cd2536d9b18faa9"
format_stamp: "Formatted at 2026-09-25 09:27:09 on dist-test-slave-9pj8"
server_key: "f01985856eb103150f7b743c216ffd02"
server_key_iv: "21be09f2d5035919b77cda38d511dbaf"
server_key_version: "encryptionkey@0"
I20260925 09:27:09.367161 16081 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:27:09.370457 16095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:09.371672 16081 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 09:27:09.371883 16081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "ed4f805faef848f38cd2536d9b18faa9"
format_stamp: "Formatted at 2026-09-25 09:27:09 on dist-test-slave-9pj8"
server_key: "f01985856eb103150f7b743c216ffd02"
server_key_iv: "21be09f2d5035919b77cda38d511dbaf"
server_key_version: "encryptionkey@0"
I20260925 09:27:09.372087 16081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:09.386929 16081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:09.387669 16081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:09.388033 16081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:09.406469 16081 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:33693
I20260925 09:27:09.406486 16146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:33693 every 8 connection(s)
I20260925 09:27:09.407977 16081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:27:09.411571 16147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:09.412335 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 16081
I20260925 09:27:09.412559 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:27:09.413190 10697 external_mini_cluster.cc:1428] Setting key da33afaf449b293f25515e160b45d728
I20260925 09:27:09.426486 16147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9: Bootstrap starting.
I20260925 09:27:09.428546 16150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:09.415141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38424 (local address 127.10.114.126:33693)
0925 09:27:09.416218 (+ 1077us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:09.416233 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:09.416279 (+ 46us) server_negotiation.cc:407] Connection header received
0925 09:27:09.418232 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:09.418259 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:09.418829 (+ 570us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:09.419294 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:09.419846 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:09.421896 (+ 2050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:09.423233 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:09.424392 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:09.424583 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:09.425695 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:09.425706 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:09.425723 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:09.426419 (+ 696us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:09.426525 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:09.426561 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:09.426671 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:09.426976 (+ 305us) server_negotiation.cc:299] Negotiation successful
0925 09:27:09.427270 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":315,"threads_started":1}
I20260925 09:27:09.430447 16147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:09.431627 16147 log.cc:824] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:09.432302 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:27:09.434299 16147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9: No bootstrap required, opened a new log
I20260925 09:27:09.442729 16147 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed4f805faef848f38cd2536d9b18faa9" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33693 } }
I20260925 09:27:09.443063 16147 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:09.443184 16147 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed4f805faef848f38cd2536d9b18faa9, State: Initialized, Role: FOLLOWER
I20260925 09:27:09.443728 16147 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed4f805faef848f38cd2536d9b18faa9" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33693 } }
I20260925 09:27:09.444022 16147 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:27:09.444185 16147 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:27:09.444367 16147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:09.446431 16147 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed4f805faef848f38cd2536d9b18faa9" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33693 } }
I20260925 09:27:09.446954 16147 leader_election.cc:302] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed4f805faef848f38cd2536d9b18faa9; no voters:
I20260925 09:27:09.447551 16147 leader_election.cc:288] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:27:09.447722 16152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:09.449050 16152 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9 [term 1 LEADER]: Becoming Leader. State: Replica: ed4f805faef848f38cd2536d9b18faa9, State: Running, Role: LEADER
I20260925 09:27:09.449513 16152 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed4f805faef848f38cd2536d9b18faa9" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33693 } }
I20260925 09:27:09.450023 16147 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:27:09.452034 16153 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed4f805faef848f38cd2536d9b18faa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed4f805faef848f38cd2536d9b18faa9" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33693 } } }
I20260925 09:27:09.452441 16153 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:09.452090 16154 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed4f805faef848f38cd2536d9b18faa9. Latest consensus state: current_term: 1 leader_uuid: "ed4f805faef848f38cd2536d9b18faa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed4f805faef848f38cd2536d9b18faa9" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33693 } } }
I20260925 09:27:09.452668 16154 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:09.454509 16160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:27:09.459568 16160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:27:09.468007 16160 catalog_manager.cc:1347] Generated new cluster ID: 52ee6939ce6b42c78140fa1b00653eac
I20260925 09:27:09.468113 16160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:27:09.478480 16160 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:27:09.480155 16160 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:27:09.509418 16160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed4f805faef848f38cd2536d9b18faa9: Generated new TSK 0
I20260925 09:27:09.510370 16160 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:27:09.538834 16150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:09.530506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38438 (local address 127.10.114.126:33693)
0925 09:27:09.530707 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:09.530716 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:09.531265 (+ 549us) server_negotiation.cc:407] Connection header received
0925 09:27:09.531487 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:09.531490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:09.531556 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:09.531685 (+ 129us) server_negotiation.cc:226] Negotiated authn=JWT
0925 09:27:09.532329 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:09.533009 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:09.534944 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:09.535366 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:09.535454 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:09.536060 (+ 606us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 09:27:09.538265 (+ 2205us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46689/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 09:27:09.538559 (+ 294us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38438: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46689/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":58}
W20260925 09:27:09.538947 16150 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38438: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46689/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260925 09:27:09.541282 16176 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:09.530513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:33693 (local address 127.0.0.1:38438)
0925 09:27:09.531154 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:09.531176 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:09.531331 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:09.531735 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:09.531741 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:09.531756 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0925 09:27:09.532182 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:09.532195 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:09.533148 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:09.533154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:09.534789 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:09.534805 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:09.535550 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:09.535557 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:09.535744 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:09.535816 (+ 72us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 09:27:09.538467 (+ 2651us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46689/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 09:27:09.540804 (+ 2337us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.114.126:33693: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46689/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":2050,"thread_start_us":340,"threads_started":1}
I20260925 09:27:09.542204 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 16081
I20260925 09:27:09.543092 16142 generic_service.cc:196] Checking for leaks (request via RPC)
E20260925 09:27:09.551183 16177 webserver.cc:533] Webserver: sslize failed:
E20260925 09:27:09.551261 16177 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 09:27:09.631824 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 16081
2026-09-25T09:27:09Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (707 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-25T09:27:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:27:09Z Disabled control of system clock
I20260925 09:27:09.710053 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:45029
--webserver_interface=127.10.114.126
--webserver_port=0
--builtin_ntp_servers=127.10.114.84:38611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:45029
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790328385296020-10697-0/testchainca.cert with env {}
W20260925 09:27:10.024677 16182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:10.024978 16182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:10.025156 16182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:10.032295 16182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:27:10.032400 16182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:10.032498 16182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:10.032610 16182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:27:10.032682 16182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 09:27:10.032788 16182 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 09:27:10.032857 16182 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260925 09:27:10.043268 16182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:45029
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:45029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790328385296020-10697-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:10.044589 16182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:10.045913 16182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:10.057937 16187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:10.057899 16188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:10.058108 16190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:10.058936 16182 server_base.cc:1034] running on GCE node
I20260925 09:27:10.059675 16182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:10.061426 16182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:10.062656 16182 hybrid_clock.cc:648] HybridClock initialized: now 1790328430062608 us; error 39 us; skew 500 ppm
I20260925 09:27:10.063022 16182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:27:10.065249 16182 webserver.cc:460] Webserver started at http://127.10.114.126:39409/ using document root <none> and password file <none>
I20260925 09:27:10.065951 16182 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:10.066040 16182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:10.066386 16182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:10.069180 16182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "a188347fed8b4ee1bd116a360e76f0e2"
format_stamp: "Formatted at 2026-09-25 09:27:10 on dist-test-slave-9pj8"
server_key: "ebad6bea451df0f91ff85b18087b6b9f"
server_key_iv: "1f990fbc80d6e351fba54e61ccea6cc2"
server_key_version: "encryptionkey@0"
I20260925 09:27:10.070013 16182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "a188347fed8b4ee1bd116a360e76f0e2"
format_stamp: "Formatted at 2026-09-25 09:27:10 on dist-test-slave-9pj8"
server_key: "ebad6bea451df0f91ff85b18087b6b9f"
server_key_iv: "1f990fbc80d6e351fba54e61ccea6cc2"
server_key_version: "encryptionkey@0"
I20260925 09:27:10.075035 16182 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260925 09:27:10.078226 16196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:10.079538 16182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:27:10.079694 16182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "a188347fed8b4ee1bd116a360e76f0e2"
format_stamp: "Formatted at 2026-09-25 09:27:10 on dist-test-slave-9pj8"
server_key: "ebad6bea451df0f91ff85b18087b6b9f"
server_key_iv: "1f990fbc80d6e351fba54e61ccea6cc2"
server_key_version: "encryptionkey@0"
I20260925 09:27:10.079967 16182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:10.100647 16182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:10.101339 16182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:10.101606 16182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:10.118536 16182 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:45029
I20260925 09:27:10.118564 16247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:45029 every 8 connection(s)
I20260925 09:27:10.119704 16182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:27:10.122972 16248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:10.125231 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 16182
I20260925 09:27:10.125464 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:27:10.126124 10697 external_mini_cluster.cc:1428] Setting key c18741c06f37dad335d27132225141b5
I20260925 09:27:10.135428 16248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2: Bootstrap starting.
I20260925 09:27:10.139122 16248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:10.140405 16248 log.cc:824] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:10.141052 16251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:10.127977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34946 (local address 127.10.114.126:45029)
0925 09:27:10.128922 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:10.128944 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:10.128988 (+ 44us) server_negotiation.cc:407] Connection header received
0925 09:27:10.131167 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:10.131197 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:10.131818 (+ 621us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:10.132246 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:10.132754 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:10.134684 (+ 1930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:10.136024 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:10.137052 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:10.137239 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:10.138253 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:10.138265 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:10.138282 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:10.138915 (+ 633us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:10.139038 (+ 123us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:10.139061 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:10.139215 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:10.139475 (+ 260us) server_negotiation.cc:299] Negotiation successful
0925 09:27:10.139815 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":296,"threads_started":1}
I20260925 09:27:10.143262 16248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2: No bootstrap required, opened a new log
I20260925 09:27:10.144490 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:27:10.150594 16248 raft_consensus.cc:348] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a188347fed8b4ee1bd116a360e76f0e2" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 45029 } }
I20260925 09:27:10.150978 16248 raft_consensus.cc:374] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:10.151106 16248 raft_consensus.cc:729] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a188347fed8b4ee1bd116a360e76f0e2, State: Initialized, Role: FOLLOWER
I20260925 09:27:10.151692 16248 consensus_queue.cc:260] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a188347fed8b4ee1bd116a360e76f0e2" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 45029 } }
I20260925 09:27:10.151995 16248 raft_consensus.cc:388] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:27:10.152184 16248 raft_consensus.cc:482] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:27:10.152359 16248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:10.154429 16248 raft_consensus.cc:504] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a188347fed8b4ee1bd116a360e76f0e2" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 45029 } }
I20260925 09:27:10.154992 16248 leader_election.cc:302] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a188347fed8b4ee1bd116a360e76f0e2; no voters:
I20260925 09:27:10.155717 16248 leader_election.cc:288] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:27:10.155884 16253 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:10.156598 16253 raft_consensus.cc:686] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2 [term 1 LEADER]: Becoming Leader. State: Replica: a188347fed8b4ee1bd116a360e76f0e2, State: Running, Role: LEADER
I20260925 09:27:10.157106 16253 consensus_queue.cc:237] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a188347fed8b4ee1bd116a360e76f0e2" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 45029 } }
I20260925 09:27:10.157788 16248 sys_catalog.cc:564] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:27:10.159940 16255 sys_catalog.cc:455] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a188347fed8b4ee1bd116a360e76f0e2. Latest consensus state: current_term: 1 leader_uuid: "a188347fed8b4ee1bd116a360e76f0e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a188347fed8b4ee1bd116a360e76f0e2" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 45029 } } }
I20260925 09:27:10.160396 16255 sys_catalog.cc:458] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:10.160189 16254 sys_catalog.cc:455] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a188347fed8b4ee1bd116a360e76f0e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a188347fed8b4ee1bd116a360e76f0e2" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 45029 } } }
I20260925 09:27:10.160506 16254 sys_catalog.cc:458] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:10.161345 16261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:27:10.166782 16261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:27:10.175598 16261 catalog_manager.cc:1347] Generated new cluster ID: 72807a8dc0ec488ab59581c1928f0726
I20260925 09:27:10.175714 16261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:27:10.190881 16261 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:27:10.192044 16261 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:27:10.200505 16261 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a188347fed8b4ee1bd116a360e76f0e2: Generated new TSK 0
I20260925 09:27:10.201220 16261 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:27:10.244843 16251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:10.211907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34950 (local address 127.10.114.126:45029)
0925 09:27:10.212137 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:10.212147 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:10.212692 (+ 545us) server_negotiation.cc:407] Connection header received
0925 09:27:10.212952 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:10.212956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:10.213017 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:10.213146 (+ 129us) server_negotiation.cc:226] Negotiated authn=JWT
0925 09:27:10.213710 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:10.214420 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:10.216427 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:10.216820 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:10.216921 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:10.217512 (+ 591us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 09:27:10.244264 (+ 26752us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 09:27:10.244624 (+ 360us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34950: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":59}
W20260925 09:27:10.244992 16251 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34950: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260925 09:27:10.245682 16277 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:10.211897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:45029 (local address 127.0.0.1:34950)
0925 09:27:10.212585 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:10.212606 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:10.212779 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:10.213197 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:10.213202 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:10.213216 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0925 09:27:10.213546 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:10.213559 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:10.214566 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:10.214573 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:10.216275 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:10.216291 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:10.216991 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:10.216999 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:10.217179 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:10.217223 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 09:27:10.244573 (+ 27350us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 09:27:10.244891 (+ 318us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.114.126:45029: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":358,"threads_started":1}
I20260925 09:27:10.247233 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 16182
I20260925 09:27:10.248452 16243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:10.337180 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 16182
2026-09-25T09:27:10Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (685 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (408 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (385 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:27:11 dist-test-slave-9pj8 krb5kdc[16292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:27:11 dist-test-slave-9pj8 krb5kdc[16292](info): set up 2 sockets
Sep 25 09:27:11 dist-test-slave-9pj8 krb5kdc[16292](info): commencing operation
krb5kdc: starting...
W20260925 09:27:12.184729 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:27:12 dist-test-slave-9pj8 krb5kdc[16292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:27:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:27:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:27:12.379401 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:43955
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:43955
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:12.695425 16305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:12.695749 16305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:12.695956 16305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:12.702987 16305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:27:12.703132 16305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:12.703251 16305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:12.703367 16305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:27:12.703459 16305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:27:12.713176 16305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:43955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:43955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:12.714449 16305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:12.715746 16305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:12.727452 16311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:12.727530 16313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:12.728004 16310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:12.728821 16305 server_base.cc:1034] running on GCE node
I20260925 09:27:12.729611 16305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:12.730935 16305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:12.732174 16305 hybrid_clock.cc:648] HybridClock initialized: now 1790328432732129 us; error 36 us; skew 500 ppm
I20260925 09:27:12.732529 16305 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:12 dist-test-slave-9pj8 krb5kdc[16292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328432, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:12.735492 16305 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:27:12.736877 16305 webserver.cc:460] Webserver started at http://127.10.114.126:34319/ using document root <none> and password file <none>
I20260925 09:27:12.737460 16305 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:12.737550 16305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:12.737776 16305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:12.740633 16305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "37b5298e24d548e282a2ea5cb26b38a3"
format_stamp: "Formatted at 2026-09-25 09:27:12 on dist-test-slave-9pj8"
server_key: "02d372117915f5d676416ca219dbe627"
server_key_iv: "31d83a829ba2a30332207029192278e4"
server_key_version: "encryptionkey@0"
I20260925 09:27:12.741309 16305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "37b5298e24d548e282a2ea5cb26b38a3"
format_stamp: "Formatted at 2026-09-25 09:27:12 on dist-test-slave-9pj8"
server_key: "02d372117915f5d676416ca219dbe627"
server_key_iv: "31d83a829ba2a30332207029192278e4"
server_key_version: "encryptionkey@0"
I20260925 09:27:12.746048 16305 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:27:12.749246 16320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:12.750532 16305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260925 09:27:12.750655 16305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "37b5298e24d548e282a2ea5cb26b38a3"
format_stamp: "Formatted at 2026-09-25 09:27:12 on dist-test-slave-9pj8"
server_key: "02d372117915f5d676416ca219dbe627"
server_key_iv: "31d83a829ba2a30332207029192278e4"
server_key_version: "encryptionkey@0"
I20260925 09:27:12.750804 16305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:12.764263 16305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:12.767944 16305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:12.768146 16305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:12.786748 16305 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:43955
I20260925 09:27:12.786826 16371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:43955 every 8 connection(s)
I20260925 09:27:12.788034 16305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:27:12.791565 16372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:12.794893 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 16305
I20260925 09:27:12.795104 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:27:12.795751 10697 external_mini_cluster.cc:1428] Setting key 28f9583b533fdffc5c6b468833f1cc0d
I20260925 09:27:12.802119 16372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3: Bootstrap starting.
I20260925 09:27:12.805554 16372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:12.806692 16372 log.cc:824] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:12.809504 16372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3: No bootstrap required, opened a new log
Sep 25 09:27:12 dist-test-slave-9pj8 krb5kdc[16292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:27:12.818569 16375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:12.797733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54522 (local address 127.10.114.126:43955)
0925 09:27:12.798466 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:12.798478 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:12.798519 (+ 41us) server_negotiation.cc:407] Connection header received
0925 09:27:12.800684 (+ 2165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:12.800707 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:12.801105 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:12.801453 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:12.803229 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:12.804659 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:12.806414 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:12.807287 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:12.807511 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:12.811476 (+ 3965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:12.811485 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:12.811501 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:12.811538 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:12.814562 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:12.815369 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:12.815374 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:12.815380 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:12.815526 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:12.815969 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:12.815972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:12.815974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:12.816441 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:12.816650 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:12.817107 (+ 457us) server_negotiation.cc:299] Negotiation successful
0925 09:27:12.817373 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":344,"threads_started":1}
I20260925 09:27:12.818951 16372 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37b5298e24d548e282a2ea5cb26b38a3" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43955 } }
I20260925 09:27:12.819427 16372 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:12.819545 16372 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37b5298e24d548e282a2ea5cb26b38a3, State: Initialized, Role: FOLLOWER
I20260925 09:27:12.820230 16372 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37b5298e24d548e282a2ea5cb26b38a3" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43955 } }
I20260925 09:27:12.820550 16372 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:27:12.820726 16372 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:27:12.820910 16372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:12.823215 16372 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37b5298e24d548e282a2ea5cb26b38a3" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43955 } }
I20260925 09:27:12.823676 16372 leader_election.cc:302] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37b5298e24d548e282a2ea5cb26b38a3; no voters:
I20260925 09:27:12.824273 16372 leader_election.cc:288] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:27:12.824370 16377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:12.825129 16377 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3 [term 1 LEADER]: Becoming Leader. State: Replica: 37b5298e24d548e282a2ea5cb26b38a3, State: Running, Role: LEADER
I20260925 09:27:12.825649 16377 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37b5298e24d548e282a2ea5cb26b38a3" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43955 } }
I20260925 09:27:12.826406 16372 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:27:12.828477 16378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37b5298e24d548e282a2ea5cb26b38a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37b5298e24d548e282a2ea5cb26b38a3" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43955 } } }
I20260925 09:27:12.828946 16378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:12.832379 16379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37b5298e24d548e282a2ea5cb26b38a3. Latest consensus state: current_term: 1 leader_uuid: "37b5298e24d548e282a2ea5cb26b38a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37b5298e24d548e282a2ea5cb26b38a3" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43955 } } }
I20260925 09:27:12.832778 16379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:12.833063 16383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:27:12.837710 16383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:27:12.847111 16383 catalog_manager.cc:1347] Generated new cluster ID: 66f15e1ad904405099e7a3aca18418a6
I20260925 09:27:12.847237 16383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:27:12.872067 16383 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:27:12.873323 16383 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:27:12.879550 16383 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37b5298e24d548e282a2ea5cb26b38a3: Generated new TSK 0
I20260925 09:27:12.880317 16383 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
I20260925 09:27:12.932015 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43955
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:13.251470 16398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:13.251721 16398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:13.251889 16398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:13.259194 16398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:13.259286 16398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:13.259390 16398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:27:13.268834 16398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:13.269948 16398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:13.271194 16398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:13.284041 16403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:13.284075 16406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:13.284129 16404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:13.284868 16398 server_base.cc:1034] running on GCE node
I20260925 09:27:13.285418 16398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:13.286347 16398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:13.287633 16398 hybrid_clock.cc:648] HybridClock initialized: now 1790328433287589 us; error 37 us; skew 500 ppm
I20260925 09:27:13.287990 16398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:13 dist-test-slave-9pj8 krb5kdc[16292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328433, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:13.291208 16398 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:27:13.292737 16398 webserver.cc:460] Webserver started at http://127.10.114.65:36415/ using document root <none> and password file <none>
I20260925 09:27:13.293398 16398 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:13.293540 16398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:13.293866 16398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:13.296833 16398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "fde6052ef71a465fad958bc33d1d7e4a"
format_stamp: "Formatted at 2026-09-25 09:27:13 on dist-test-slave-9pj8"
server_key: "77d3a2eaee3eb4c21d84a720ac41f594"
server_key_iv: "2b4c54be21e2a0af8a62b768466d97ea"
server_key_version: "encryptionkey@0"
I20260925 09:27:13.297634 16398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "fde6052ef71a465fad958bc33d1d7e4a"
format_stamp: "Formatted at 2026-09-25 09:27:13 on dist-test-slave-9pj8"
server_key: "77d3a2eaee3eb4c21d84a720ac41f594"
server_key_iv: "2b4c54be21e2a0af8a62b768466d97ea"
server_key_version: "encryptionkey@0"
I20260925 09:27:13.302623 16398 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260925 09:27:13.305989 16413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:13.307183 16398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 09:27:13.307386 16398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "fde6052ef71a465fad958bc33d1d7e4a"
format_stamp: "Formatted at 2026-09-25 09:27:13 on dist-test-slave-9pj8"
server_key: "77d3a2eaee3eb4c21d84a720ac41f594"
server_key_iv: "2b4c54be21e2a0af8a62b768466d97ea"
server_key_version: "encryptionkey@0"
I20260925 09:27:13.307569 16398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:13.337589 16398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:13.341996 16398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:13.342271 16398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:13.343168 16398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:13.344578 16398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:13.344653 16398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:13.344810 16398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:13.344887 16398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:13.377013 16398 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:46869
I20260925 09:27:13.377091 16525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:46869 every 8 connection(s)
I20260925 09:27:13.378229 16398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:27:13.380030 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 16398
I20260925 09:27:13.380220 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:27:13.380859 10697 external_mini_cluster.cc:1428] Setting key 5df988c0c4149ee837ae8d0a866bdfbe
Sep 25 09:27:13 dist-test-slave-9pj8 krb5kdc[16292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328433, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.114.66@KRBTEST.COM; defaulting to no policy
I20260925 09:27:13.399731 16528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:13.381633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:55781 (local address 127.10.114.126:43955)
0925 09:27:13.382114 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:13.382120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:13.383109 (+ 989us) server_negotiation.cc:407] Connection header received
0925 09:27:13.385051 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:13.385057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:13.385149 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:13.385275 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:13.386727 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:13.387489 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:13.389829 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:13.390255 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:13.390410 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:13.393911 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:13.393916 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:13.393919 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:13.393949 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:13.396629 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:13.397360 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:13.397373 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:13.397376 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:13.397493 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:13.397950 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:13.397956 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:13.397961 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:13.398219 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:13.398390 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:13.399096 (+ 706us) server_negotiation.cc:299] Negotiation successful
0925 09:27:13.399331 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":244,"threads_started":1}
I20260925 09:27:13.400765 16529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:13.382110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43955 (local address 127.10.114.65:55781)
0925 09:27:13.382924 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:13.382995 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:13.384777 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:13.385514 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:13.385523 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:13.385998 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:13.386556 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:13.386573 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:13.387670 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:13.387675 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:13.389675 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:13.389687 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:13.390488 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:13.390498 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:13.390764 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:13.391459 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:13.391482 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:13.393686 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:13.396790 (+ 3104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:13.396800 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:13.396814 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:13.397223 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:13.397619 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:13.397624 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:13.397628 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:13.397822 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:13.398375 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:13.398383 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:13.398701 (+ 318us) client_negotiation.cc:769] Sending connection context
0925 09:27:13.398969 (+ 268us) client_negotiation.cc:240] Negotiation successful
0925 09:27:13.399466 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":321,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.66@KRBTEST.COM" created.
I20260925 09:27:13.402473 16526 heartbeater.cc:347] Connected to a master server at 127.10.114.126:43955
I20260925 09:27:13.402762 16526 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:13.403450 16526 heartbeater.cc:511] Master 127.10.114.126:43955 requested a full tablet report, sending...
I20260925 09:27:13.405599 16337 ts_manager.cc:194] Registered new tserver with Master: fde6052ef71a465fad958bc33d1d7e4a (127.10.114.65:46869)
I20260925 09:27:13.407003 16337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:55781
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
I20260925 09:27:13.426435 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43955
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:13.738368 16532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:13.738637 16532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:13.738734 16532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:13.745923 16532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:13.746027 16532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:13.746147 16532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:27:13.755213 16532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:13.756353 16532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:13.757550 16532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:13.770179 16537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:13.770252 16540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:13.770289 16538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:13.771082 16532 server_base.cc:1034] running on GCE node
I20260925 09:27:13.771678 16532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:13.772632 16532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:13.773865 16532 hybrid_clock.cc:648] HybridClock initialized: now 1790328433773821 us; error 58 us; skew 500 ppm
I20260925 09:27:13.774164 16532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:13 dist-test-slave-9pj8 krb5kdc[16292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328433, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:13.777110 16532 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:27:13.778421 16532 webserver.cc:460] Webserver started at http://127.10.114.66:35961/ using document root <none> and password file <none>
I20260925 09:27:13.778975 16532 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:13.779098 16532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:13.779345 16532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:13.782063 16532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "019e05c60713439ea21d4d908f1670fc"
format_stamp: "Formatted at 2026-09-25 09:27:13 on dist-test-slave-9pj8"
server_key: "5e554ec4bcc88dbc7a27c7c6b6b0240f"
server_key_iv: "c83c3d3fb3a14fa1773a4455afdee77d"
server_key_version: "encryptionkey@0"
I20260925 09:27:13.782763 16532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "019e05c60713439ea21d4d908f1670fc"
format_stamp: "Formatted at 2026-09-25 09:27:13 on dist-test-slave-9pj8"
server_key: "5e554ec4bcc88dbc7a27c7c6b6b0240f"
server_key_iv: "c83c3d3fb3a14fa1773a4455afdee77d"
server_key_version: "encryptionkey@0"
I20260925 09:27:13.789286 16532 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.008s
I20260925 09:27:13.792768 16547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:13.794021 16532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:27:13.794180 16532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "019e05c60713439ea21d4d908f1670fc"
format_stamp: "Formatted at 2026-09-25 09:27:13 on dist-test-slave-9pj8"
server_key: "5e554ec4bcc88dbc7a27c7c6b6b0240f"
server_key_iv: "c83c3d3fb3a14fa1773a4455afdee77d"
server_key_version: "encryptionkey@0"
I20260925 09:27:13.794395 16532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:13.830807 16532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:13.834450 16532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:13.834733 16532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:13.835686 16532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:13.836923 16532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:13.837049 16532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:13.837194 16532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:13.837266 16532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:13.869709 16532 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:34947
I20260925 09:27:13.869796 16659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:34947 every 8 connection(s)
I20260925 09:27:13.870919 16532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:27:13.873001 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 16532
I20260925 09:27:13.873170 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:27:13.873675 10697 external_mini_cluster.cc:1428] Setting key 747f64ee96e2a796500dedec9c9a0e25
Sep 25 09:27:13 dist-test-slave-9pj8 krb5kdc[16292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328433, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:27:13.893007 16528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:13.873392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:37563 (local address 127.10.114.126:43955)
0925 09:27:13.873552 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:13.873557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:13.875099 (+ 1542us) server_negotiation.cc:407] Connection header received
0925 09:27:13.877654 (+ 2555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:13.877658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:13.877723 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:13.877809 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:13.879963 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:13.880685 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:13.883015 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:13.883749 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:13.883928 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:13.887070 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:13.887075 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:13.887078 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:13.887107 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:13.889668 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:13.890446 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:13.890454 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:13.890460 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:13.890626 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:13.891089 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:13.891097 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:13.891104 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:13.891506 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:13.891680 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:13.892399 (+ 719us) server_negotiation.cc:299] Negotiation successful
0925 09:27:13.892633 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 09:27:13.893716 16662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:13.874029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43955 (local address 127.10.114.66:37563)
0925 09:27:13.874939 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:13.874986 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:13.877340 (+ 2354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:13.878087 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:13.878103 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:13.878856 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:13.879689 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:13.879716 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:13.880865 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:13.880873 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:13.882870 (+ 1997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:13.882882 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:13.883981 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:13.883986 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:13.884177 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:13.884854 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:13.884874 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:13.886862 (+ 1988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:13.889844 (+ 2982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:13.889854 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:13.889870 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:13.890290 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:13.890786 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:13.890790 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:13.890792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:13.890976 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:13.891646 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:13.891652 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:13.892020 (+ 368us) client_negotiation.cc:769] Sending connection context
0925 09:27:13.892252 (+ 232us) client_negotiation.cc:240] Negotiation successful
0925 09:27:13.892609 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":397,"threads_started":1}
WARNING: no policy specified for kudu/127.10.114.67@KRBTEST.COM; defaulting to no policy
I20260925 09:27:13.895288 16660 heartbeater.cc:347] Connected to a master server at 127.10.114.126:43955
I20260925 09:27:13.895574 16660 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:13.896334 16660 heartbeater.cc:511] Master 127.10.114.126:43955 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.67@KRBTEST.COM" created.
I20260925 09:27:13.897852 16337 ts_manager.cc:194] Registered new tserver with Master: 019e05c60713439ea21d4d908f1670fc (127.10.114.66:34947)
I20260925 09:27:13.899116 16337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:37563
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
I20260925 09:27:13.921221 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43955
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:14.229486 16665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:14.229799 16665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:14.229971 16665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:14.236968 16665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:14.237108 16665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:14.237267 16665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:27:14.246860 16665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:14.248093 16665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:14.249348 16665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:14.261799 16673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:14.261759 16671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:14.261869 16670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:14.262549 16665 server_base.cc:1034] running on GCE node
I20260925 09:27:14.263150 16665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:14.264057 16665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:14.265267 16665 hybrid_clock.cc:648] HybridClock initialized: now 1790328434265228 us; error 51 us; skew 500 ppm
I20260925 09:27:14.265547 16665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:14 dist-test-slave-9pj8 krb5kdc[16292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328434, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:14.268476 16665 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:27:14.269881 16665 webserver.cc:460] Webserver started at http://127.10.114.67:38655/ using document root <none> and password file <none>
I20260925 09:27:14.270543 16665 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:14.270681 16665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:14.270977 16665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:14.273950 16665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "9a7d6ea73da4410ca600fad1be5d6fa8"
format_stamp: "Formatted at 2026-09-25 09:27:14 on dist-test-slave-9pj8"
server_key: "e5b3e4e0d9bf1efd40121118f907a1ee"
server_key_iv: "a5003fa611228cddeaf27303e14fd21b"
server_key_version: "encryptionkey@0"
I20260925 09:27:14.274709 16665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "9a7d6ea73da4410ca600fad1be5d6fa8"
format_stamp: "Formatted at 2026-09-25 09:27:14 on dist-test-slave-9pj8"
server_key: "e5b3e4e0d9bf1efd40121118f907a1ee"
server_key_iv: "a5003fa611228cddeaf27303e14fd21b"
server_key_version: "encryptionkey@0"
I20260925 09:27:14.279664 16665 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260925 09:27:14.282961 16680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:14.284279 16665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:27:14.284456 16665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "9a7d6ea73da4410ca600fad1be5d6fa8"
format_stamp: "Formatted at 2026-09-25 09:27:14 on dist-test-slave-9pj8"
server_key: "e5b3e4e0d9bf1efd40121118f907a1ee"
server_key_iv: "a5003fa611228cddeaf27303e14fd21b"
server_key_version: "encryptionkey@0"
I20260925 09:27:14.284657 16665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:14.311090 16665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:14.315114 16665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:14.315434 16665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:14.316497 16665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:14.317835 16665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:14.317989 16665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:14.318150 16665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:14.318229 16665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:14.350345 16665 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:40471
I20260925 09:27:14.350421 16792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:40471 every 8 connection(s)
I20260925 09:27:14.351490 16665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:27:14.357093 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 16665
I20260925 09:27:14.357288 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:27:14.357772 10697 external_mini_cluster.cc:1428] Setting key cf99cecaf39534d76a383b32d32d8bc4
Sep 25 09:27:14 dist-test-slave-9pj8 krb5kdc[16292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328434, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:27:14.374684 16528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:14.353799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:36121 (local address 127.10.114.126:43955)
0925 09:27:14.353985 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:14.353991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:14.355384 (+ 1393us) server_negotiation.cc:407] Connection header received
0925 09:27:14.357287 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:14.357293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:14.357380 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:14.357527 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:14.359205 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:14.360294 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:14.361778 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:14.362141 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:14.362235 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:14.369056 (+ 6821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:14.369060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:14.369063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:14.369092 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:14.372042 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:14.372817 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:14.372822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:14.372826 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:14.373002 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:14.373379 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:14.373382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:14.373384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:14.373635 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:14.373744 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:14.374298 (+ 554us) server_negotiation.cc:299] Negotiation successful
0925 09:27:14.374444 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:27:14.375602 16795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:14.354292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43955 (local address 127.10.114.67:36121)
0925 09:27:14.355195 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:14.355269 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:14.356972 (+ 1703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:14.357747 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:14.357755 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:14.358336 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:14.359010 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:14.359029 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:14.360450 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:14.360456 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:14.361647 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:14.361662 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:14.362316 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:14.362320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:14.362502 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:14.366830 (+ 4328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:14.366850 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:14.368872 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:14.372176 (+ 3304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:14.372184 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:14.372199 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:14.372683 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:14.373120 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:14.373123 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:14.373125 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:14.373282 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:14.373737 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:14.373742 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:14.374003 (+ 261us) client_negotiation.cc:769] Sending connection context
0925 09:27:14.374209 (+ 206us) client_negotiation.cc:240] Negotiation successful
0925 09:27:14.374528 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":344,"threads_started":1}
I20260925 09:27:14.377022 16793 heartbeater.cc:347] Connected to a master server at 127.10.114.126:43955
I20260925 09:27:14.377346 16793 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:14.378000 16793 heartbeater.cc:511] Master 127.10.114.126:43955 requested a full tablet report, sending...
I20260925 09:27:14.379417 16337 ts_manager.cc:194] Registered new tserver with Master: 9a7d6ea73da4410ca600fad1be5d6fa8 (127.10.114.67:40471)
I20260925 09:27:14.380374 16337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:36121
I20260925 09:27:14.384977 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:27:14.410593 16526 heartbeater.cc:503] Master 127.10.114.126:43955 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/test-admin.keytab.
Sep 25 09:27:14 dist-test-slave-9pj8 krb5kdc[16292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:14.414232 10697 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260925 09:27:14.418924 10697 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/krb5cc' of type FILE
W20260925 09:27:14.418955 16800 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:14.416645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33736 (local address 127.10.114.65:46869)
0925 09:27:14.417292 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:14.417298 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:14.417324 (+ 26us) server_negotiation.cc:407] Connection header received
0925 09:27:14.417375 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:14.417381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:14.417581 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:14.417703 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:14.418521 (+ 818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33736: BlockingRecv error: recv got EOF from 127.0.0.1:33736 (error 108)
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":354,"threads_started":1}
Sep 25 09:27:14 dist-test-slave-9pj8 krb5kdc[16292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:14.420491 10697 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 09:27:14 dist-test-slave-9pj8 krb5kdc[16292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:14.421839 10697 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 09:27:14 dist-test-slave-9pj8 krb5kdc[16292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:27:14.436798 16800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:14.423331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33746 (local address 127.10.114.65:46869)
0925 09:27:14.423509 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:14.423515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:14.423696 (+ 181us) server_negotiation.cc:407] Connection header received
0925 09:27:14.423945 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:14.423948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:14.423998 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:14.424101 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:14.424991 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:14.425924 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:14.427163 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:14.427679 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:14.427832 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:14.431168 (+ 3336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:14.431179 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:14.431191 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:14.431219 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:14.433916 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:14.434594 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:14.434603 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:14.434605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:14.434748 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:14.435128 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:14.435136 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:14.435139 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:14.435489 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:14.435816 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:14.436228 (+ 412us) server_negotiation.cc:299] Negotiation successful
0925 09:27:14.436524 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260925 09:27:14.442195 16800 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:14.440858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33750 (local address 127.10.114.65:46869)
0925 09:27:14.441033 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:14.441038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:14.441183 (+ 145us) server_negotiation.cc:407] Connection header received
0925 09:27:14.441315 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:14.441318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:14.441364 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:14.441435 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:14.442018 (+ 583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33750: BlockingRecv error: recv got EOF from 127.0.0.1:33750 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260925 09:27:14.442415 10697 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/krb5cc' of type FILE
Sep 25 09:27:14 dist-test-slave-9pj8 krb5kdc[16292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:14.443851 10697 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 09:27:14 dist-test-slave-9pj8 krb5kdc[16292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:14.445072 10697 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 09:27:14 dist-test-slave-9pj8 krb5kdc[16292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:27:14.458274 16800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:14.446170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33752 (local address 127.10.114.65:46869)
0925 09:27:14.446347 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:14.446355 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:14.446631 (+ 276us) server_negotiation.cc:407] Connection header received
0925 09:27:14.446796 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:14.446806 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:14.446852 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:14.446944 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:14.447824 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:14.448412 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:14.449510 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:14.449922 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:14.450057 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:14.453230 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:14.453235 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:14.453238 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:14.453265 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:14.456029 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:14.456581 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:14.456587 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:14.456604 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:14.456706 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:14.457081 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:14.457087 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:14.457090 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:14.457333 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:14.457585 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:14.457856 (+ 271us) server_negotiation.cc:299] Negotiation successful
0925 09:27:14.458043 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260925 09:27:14.462421 16800 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:14.460770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33756 (local address 127.10.114.65:46869)
0925 09:27:14.460979 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:14.460985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:14.461231 (+ 246us) server_negotiation.cc:407] Connection header received
0925 09:27:14.461456 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:14.461459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:14.461505 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:14.461600 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:14.462207 (+ 607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33756: BlockingRecv error: recv got EOF from 127.0.0.1:33756 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260925 09:27:14.462913 10697 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790328385296020-10697-0/krb5kdc/krb5cc)
Sep 25 09:27:14 dist-test-slave-9pj8 krb5kdc[16292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:14.464356 10697 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 09:27:14 dist-test-slave-9pj8 krb5kdc[16292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:14.465687 10697 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 09:27:14 dist-test-slave-9pj8 krb5kdc[16292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:27:14.479015 16800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:14.466853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33758 (local address 127.10.114.65:46869)
0925 09:27:14.467039 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:14.467056 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:14.467285 (+ 229us) server_negotiation.cc:407] Connection header received
0925 09:27:14.467475 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:14.467478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:14.467528 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:14.467659 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:14.468755 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:14.469492 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:14.470721 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:14.471147 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:14.471242 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:14.474607 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:14.474612 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:14.474615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:14.474641 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:14.477036 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:14.477586 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:14.477589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:14.477591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:14.477697 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:14.477994 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:14.477997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:14.477999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:14.478227 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:14.478467 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:14.478615 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 09:27:14.478753 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:27:14.480538 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 16398
I20260925 09:27:14.491699 16800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:14.481124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33770 (local address 127.10.114.65:46869)
0925 09:27:14.481255 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:14.481259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:14.481541 (+ 282us) server_negotiation.cc:407] Connection header received
0925 09:27:14.481638 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:14.481641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:14.481684 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:14.481752 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:14.482514 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:14.483121 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:14.484515 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:14.484961 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:14.485063 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:14.486944 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:14.486949 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:14.486952 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:14.486991 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:14.489548 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:14.490143 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:14.490147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:14.490150 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:14.490291 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:14.490614 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:14.490617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:14.490619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:14.490865 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:14.491073 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:14.491305 (+ 232us) server_negotiation.cc:299] Negotiation successful
0925 09:27:14.491449 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 09:27:14.492269 16521 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:14.587118 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 16398
I20260925 09:27:14.605159 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 16532
Sep 25 09:27:14 dist-test-slave-9pj8 krb5kdc[16292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.66@KRBTEST.COM
I20260925 09:27:14.620584 16813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:14.606074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45038 (local address 127.10.114.66:34947)
0925 09:27:14.606729 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:14.606736 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:14.606762 (+ 26us) server_negotiation.cc:407] Connection header received
0925 09:27:14.606833 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:14.606840 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:14.607048 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:14.607173 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:14.608249 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:14.609306 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:14.610586 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:14.611127 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:14.611221 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:14.614960 (+ 3739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:14.614970 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:14.614984 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:14.615022 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:14.617869 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:14.618533 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:14.618540 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:14.618543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:14.618655 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:14.619019 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:14.619022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:14.619024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:14.619333 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:14.619556 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:14.619994 (+ 438us) server_negotiation.cc:299] Negotiation successful
0925 09:27:14.620183 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":387,"threads_started":1}
I20260925 09:27:14.621825 16655 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:14.702991 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 16532
I20260925 09:27:14.718704 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 16665
Sep 25 09:27:14 dist-test-slave-9pj8 krb5kdc[16292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.67@KRBTEST.COM
I20260925 09:27:14.734261 16815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:14.719713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38398 (local address 127.10.114.67:40471)
0925 09:27:14.720412 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:14.720419 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:14.720448 (+ 29us) server_negotiation.cc:407] Connection header received
0925 09:27:14.720504 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:14.720511 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:14.720715 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:14.720899 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:14.721935 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:14.722883 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:14.724164 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:14.724699 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:14.724873 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:14.728635 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:14.728645 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:14.728657 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:14.728687 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:14.731309 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:14.732142 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:14.732152 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:14.732156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:14.732294 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:14.732707 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:14.732713 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:14.732718 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:14.733071 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:14.733404 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:14.733707 (+ 303us) server_negotiation.cc:299] Negotiation successful
0925 09:27:14.733972 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":396,"threads_started":1}
I20260925 09:27:14.735531 16788 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:14.822813 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 16665
I20260925 09:27:14.838141 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 16305
I20260925 09:27:14.839356 16367 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:14.937922 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 16305
2026-09-25T09:27:14Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3811 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:27:15 dist-test-slave-9pj8 krb5kdc[16820](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:27:15 dist-test-slave-9pj8 krb5kdc[16820](info): set up 2 sockets
Sep 25 09:27:15 dist-test-slave-9pj8 krb5kdc[16820](info): commencing operation
krb5kdc: starting...
W20260925 09:27:16.000458 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:27:16 dist-test-slave-9pj8 krb5kdc[16820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:27:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:27:16Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.126.keytab.
Entry for principal oryx/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.126.keytab.
I20260925 09:27:16.187343 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:43419
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.126.keytab
--principal=oryx/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:39685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:43419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:16.492798 16833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:16.493247 16833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:16.493427 16833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:16.502923 16833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:27:16.503054 16833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:16.503139 16833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:16.503253 16833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:27:16.503333 16833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:27:16.515944 16833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:39685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:43419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.126.keytab
--principal=oryx/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:43419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:16.517199 16833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:16.518867 16833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:16.531418 16841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:16.531451 16839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:16.531505 16838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:16.532452 16833 server_base.cc:1034] running on GCE node
I20260925 09:27:16.533190 16833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:16.534556 16833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:16.535846 16833 hybrid_clock.cc:648] HybridClock initialized: now 1790328436535742 us; error 42 us; skew 500 ppm
I20260925 09:27:16.536197 16833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:16 dist-test-slave-9pj8 krb5kdc[16820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328436, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:16.539695 16833 init.cc:377] Logged in from keytab as oryx/127.10.114.126@KRBTEST.COM (short username oryx)
I20260925 09:27:16.541213 16833 webserver.cc:460] Webserver started at http://127.10.114.126:44343/ using document root <none> and password file <none>
I20260925 09:27:16.541882 16833 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:16.542027 16833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:16.542330 16833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:16.545606 16833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "de3ad4b7da3040669e2c44a8dfe9cf04"
format_stamp: "Formatted at 2026-09-25 09:27:16 on dist-test-slave-9pj8"
server_key: "ab0bd868142cf4a2d34f26ae8a221a9d"
server_key_iv: "396f7c6f654fa3c395542aa0b5d416fe"
server_key_version: "encryptionkey@0"
I20260925 09:27:16.546610 16833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "de3ad4b7da3040669e2c44a8dfe9cf04"
format_stamp: "Formatted at 2026-09-25 09:27:16 on dist-test-slave-9pj8"
server_key: "ab0bd868142cf4a2d34f26ae8a221a9d"
server_key_iv: "396f7c6f654fa3c395542aa0b5d416fe"
server_key_version: "encryptionkey@0"
I20260925 09:27:16.551852 16833 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260925 09:27:16.555116 16848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:16.556427 16833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:27:16.556598 16833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "de3ad4b7da3040669e2c44a8dfe9cf04"
format_stamp: "Formatted at 2026-09-25 09:27:16 on dist-test-slave-9pj8"
server_key: "ab0bd868142cf4a2d34f26ae8a221a9d"
server_key_iv: "396f7c6f654fa3c395542aa0b5d416fe"
server_key_version: "encryptionkey@0"
I20260925 09:27:16.556753 16833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:16.573592 16833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:16.577739 16833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:16.578023 16833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:16.594944 16833 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:43419
I20260925 09:27:16.594972 16899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:43419 every 8 connection(s)
I20260925 09:27:16.596287 16833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:27:16.599733 16900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:16.602501 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 16833
I20260925 09:27:16.602700 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:27:16.603493 10697 external_mini_cluster.cc:1428] Setting key 8121f2423e06de88f9650c84a00830b7
I20260925 09:27:16.611809 16900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04: Bootstrap starting.
I20260925 09:27:16.615291 16900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:16.616564 16900 log.cc:824] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04: Log is configured to *not* fsync() on all Append() calls
Sep 25 09:27:16 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM
I20260925 09:27:16.619546 16900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04: No bootstrap required, opened a new log
I20260925 09:27:16.625715 16903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:16.605455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52542 (local address 127.10.114.126:43419)
0925 09:27:16.606156 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:16.606166 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:16.606204 (+ 38us) server_negotiation.cc:407] Connection header received
0925 09:27:16.607848 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:16.607868 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:16.608247 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:16.608554 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:16.610300 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:16.611501 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:16.613216 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:16.613861 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:16.613981 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:16.618806 (+ 4825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:16.618813 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:16.618831 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:16.618865 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:16.621999 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:16.622745 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:16.622751 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:16.622756 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:16.622914 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:16.623301 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:16.623304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:16.623306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:16.623735 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:16.624045 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:16.624439 (+ 394us) server_negotiation.cc:299] Negotiation successful
0925 09:27:16.624763 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":328,"threads_started":1}
I20260925 09:27:16.628445 16900 raft_consensus.cc:348] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } }
I20260925 09:27:16.628849 16900 raft_consensus.cc:374] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:16.628947 16900 raft_consensus.cc:729] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de3ad4b7da3040669e2c44a8dfe9cf04, State: Initialized, Role: FOLLOWER
I20260925 09:27:16.629443 16900 consensus_queue.cc:260] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } }
I20260925 09:27:16.629668 16900 raft_consensus.cc:388] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:27:16.629812 16900 raft_consensus.cc:482] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:27:16.629980 16900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:16.632292 16900 raft_consensus.cc:504] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } }
I20260925 09:27:16.632743 16900 leader_election.cc:302] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de3ad4b7da3040669e2c44a8dfe9cf04; no voters:
I20260925 09:27:16.633267 16900 leader_election.cc:288] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:27:16.633401 16905 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:16.634649 16905 raft_consensus.cc:686] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [term 1 LEADER]: Becoming Leader. State: Replica: de3ad4b7da3040669e2c44a8dfe9cf04, State: Running, Role: LEADER
I20260925 09:27:16.635161 16905 consensus_queue.cc:237] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } }
I20260925 09:27:16.635339 16900 sys_catalog.cc:564] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:27:16.637487 16906 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } }
I20260925 09:27:16.637774 16906 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:16.638024 16907 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de3ad4b7da3040669e2c44a8dfe9cf04. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } }
I20260925 09:27:16.638249 16907 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:16.639950 16912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:27:16.644452 16912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:27:16.652139 16912 catalog_manager.cc:1347] Generated new cluster ID: 5344fdedc2704f5782bd1313986aaf4e
I20260925 09:27:16.652246 16912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:27:16.674266 16912 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:27:16.675403 16912 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:27:16.685285 16912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04: Generated new TSK 0
I20260925 09:27:16.685936 16912 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.65.keytab.
Entry for principal oryx/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.65.keytab.
I20260925 09:27:16.732833 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43419
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.65.keytab
--principal=oryx/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:39685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:17.043440 16926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:17.043722 16926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:17.043903 16926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:17.050782 16926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:17.050913 16926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:17.051069 16926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:27:17.059811 16926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:39685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.65.keytab
--principal=oryx/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:17.060964 16926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:17.062314 16926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:17.074571 16934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:17.074627 16932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:17.075321 16931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:17.075734 16926 server_base.cc:1034] running on GCE node
I20260925 09:27:17.076296 16926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:17.077234 16926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:17.078418 16926 hybrid_clock.cc:648] HybridClock initialized: now 1790328437078367 us; error 52 us; skew 500 ppm
I20260925 09:27:17.078696 16926 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:17 dist-test-slave-9pj8 krb5kdc[16820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328437, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:17.081815 16926 init.cc:377] Logged in from keytab as oryx/127.10.114.65@KRBTEST.COM (short username oryx)
I20260925 09:27:17.083132 16926 webserver.cc:460] Webserver started at http://127.10.114.65:35697/ using document root <none> and password file <none>
I20260925 09:27:17.083746 16926 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:17.083958 16926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:17.084237 16926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:17.086906 16926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "5cd99001c7384aaab17cb23748b9611a"
format_stamp: "Formatted at 2026-09-25 09:27:17 on dist-test-slave-9pj8"
server_key: "cfae12940f3cf6d549dc107b0e63778b"
server_key_iv: "25c9aab81c88662e329acf229f640451"
server_key_version: "encryptionkey@0"
I20260925 09:27:17.087656 16926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "5cd99001c7384aaab17cb23748b9611a"
format_stamp: "Formatted at 2026-09-25 09:27:17 on dist-test-slave-9pj8"
server_key: "cfae12940f3cf6d549dc107b0e63778b"
server_key_iv: "25c9aab81c88662e329acf229f640451"
server_key_version: "encryptionkey@0"
I20260925 09:27:17.092329 16926 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:27:17.095372 16941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:17.096607 16926 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 09:27:17.096765 16926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "5cd99001c7384aaab17cb23748b9611a"
format_stamp: "Formatted at 2026-09-25 09:27:17 on dist-test-slave-9pj8"
server_key: "cfae12940f3cf6d549dc107b0e63778b"
server_key_iv: "25c9aab81c88662e329acf229f640451"
server_key_version: "encryptionkey@0"
I20260925 09:27:17.096966 16926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:17.119444 16926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:17.123574 16926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:17.123950 16926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:17.124934 16926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:17.126463 16926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:17.126539 16926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:17.126652 16926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:17.126721 16926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:17.159675 16926 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:38265
I20260925 09:27:17.159718 17053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:38265 every 8 connection(s)
I20260925 09:27:17.160938 16926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:27:17.171101 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 16926
I20260925 09:27:17.171334 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:27:17.172047 10697 external_mini_cluster.cc:1428] Setting key e58438be2516dcff63f63a5124495da1
Sep 25 09:27:17 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328437, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM
I20260925 09:27:17.180042 17056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:17.163288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:39833 (local address 127.10.114.126:43419)
0925 09:27:17.163695 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:17.163701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:17.164821 (+ 1120us) server_negotiation.cc:407] Connection header received
0925 09:27:17.166533 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:17.166538 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:17.166599 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:17.166688 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:17.168119 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:17.168842 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:17.170218 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:17.170590 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:17.170745 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:17.174321 (+ 3576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:17.174326 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:17.174329 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:17.174364 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:17.176760 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:17.177586 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:17.177593 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:17.177598 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:17.177723 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:17.178151 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:17.178157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:17.178161 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:17.178456 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:17.178574 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:17.179405 (+ 831us) server_negotiation.cc:299] Negotiation successful
0925 09:27:17.179612 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":204,"threads_started":1}
I20260925 09:27:17.181190 17057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:17.163975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.65:39833)
0925 09:27:17.164655 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:17.164702 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:17.166309 (+ 1607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:17.166885 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:17.166892 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:17.167351 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:17.167959 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:17.167977 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:17.168956 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:17.168959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:17.170100 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:17.170110 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:17.170754 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:17.170759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:17.170942 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:17.171675 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:17.171694 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:17.174119 (+ 2425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:17.176891 (+ 2772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:17.176902 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:17.176918 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:17.177477 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:17.177828 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:17.177835 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:17.177840 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:17.178050 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:17.178573 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:17.178582 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:17.179031 (+ 449us) client_negotiation.cc:769] Sending connection context
0925 09:27:17.179297 (+ 266us) client_negotiation.cc:240] Negotiation successful
0925 09:27:17.179835 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":278,"threads_started":1}
I20260925 09:27:17.182519 17054 heartbeater.cc:347] Connected to a master server at 127.10.114.126:43419
I20260925 09:27:17.182813 17054 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:17.183439 17054 heartbeater.cc:511] Master 127.10.114.126:43419 requested a full tablet report, sending...
I20260925 09:27:17.185593 16865 ts_manager.cc:194] Registered new tserver with Master: 5cd99001c7384aaab17cb23748b9611a (127.10.114.65:38265)
I20260925 09:27:17.186949 16865 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:39833
WARNING: no policy specified for oryx/127.10.114.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.114.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.66.keytab.
Entry for principal oryx/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.66.keytab.
I20260925 09:27:17.216651 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43419
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.66.keytab
--principal=oryx/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:39685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:17.524178 17060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:17.524583 17060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:17.524739 17060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:17.531597 17060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:17.531679 17060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:17.531831 17060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:27:17.540997 17060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:39685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.66.keytab
--principal=oryx/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:17.542223 17060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:17.543442 17060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:17.555513 17065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:17.555552 17068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:17.555575 17066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:17.556731 17060 server_base.cc:1034] running on GCE node
I20260925 09:27:17.557293 17060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:17.558087 17060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:17.559304 17060 hybrid_clock.cc:648] HybridClock initialized: now 1790328437559266 us; error 45 us; skew 500 ppm
I20260925 09:27:17.559587 17060 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:17 dist-test-slave-9pj8 krb5kdc[16820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328437, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:17.562731 17060 init.cc:377] Logged in from keytab as oryx/127.10.114.66@KRBTEST.COM (short username oryx)
I20260925 09:27:17.564159 17060 webserver.cc:460] Webserver started at http://127.10.114.66:34287/ using document root <none> and password file <none>
I20260925 09:27:17.564782 17060 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:17.564929 17060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:17.565212 17060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:17.567893 17060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "32afe2fd0d664cb588b6b4a2740e1106"
format_stamp: "Formatted at 2026-09-25 09:27:17 on dist-test-slave-9pj8"
server_key: "715b38b0a376311a799e27ab00ec28ae"
server_key_iv: "cd848415eb9a74d668a9039e953d5938"
server_key_version: "encryptionkey@0"
I20260925 09:27:17.568616 17060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "32afe2fd0d664cb588b6b4a2740e1106"
format_stamp: "Formatted at 2026-09-25 09:27:17 on dist-test-slave-9pj8"
server_key: "715b38b0a376311a799e27ab00ec28ae"
server_key_iv: "cd848415eb9a74d668a9039e953d5938"
server_key_version: "encryptionkey@0"
I20260925 09:27:17.573267 17060 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:27:17.576347 17075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:17.577523 17060 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260925 09:27:17.577682 17060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "32afe2fd0d664cb588b6b4a2740e1106"
format_stamp: "Formatted at 2026-09-25 09:27:17 on dist-test-slave-9pj8"
server_key: "715b38b0a376311a799e27ab00ec28ae"
server_key_iv: "cd848415eb9a74d668a9039e953d5938"
server_key_version: "encryptionkey@0"
I20260925 09:27:17.577862 17060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:17.591176 17060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:17.595494 17060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:17.595755 17060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:17.596783 17060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:17.598232 17060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:17.598335 17060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:17.598438 17060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:17.598493 17060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:17.630542 17060 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:36841
I20260925 09:27:17.630585 17187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:36841 every 8 connection(s)
I20260925 09:27:17.631705 17060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:27:17.633051 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 17060
I20260925 09:27:17.633220 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:27:17.633718 10697 external_mini_cluster.cc:1428] Setting key 5b71129a895c1b3053b40d812ac60284
Sep 25 09:27:17 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328437, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM
I20260925 09:27:17.653195 17056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:17.634338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:60169 (local address 127.10.114.126:43419)
0925 09:27:17.634540 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:17.634548 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:17.636010 (+ 1462us) server_negotiation.cc:407] Connection header received
0925 09:27:17.638851 (+ 2841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:17.638855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:17.638916 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:17.639010 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:17.641032 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:17.641890 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:17.643718 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:17.644198 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:17.644365 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:17.647669 (+ 3304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:17.647673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:17.647676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:17.647705 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:17.650246 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:17.650943 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:17.650950 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:17.650953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:17.651077 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:17.651541 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:17.651548 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:17.651551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:17.651846 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:17.652007 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:17.652610 (+ 603us) server_negotiation.cc:299] Negotiation successful
0925 09:27:17.652843 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 09:27:17.654134 17190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:17.634808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.66:60169)
0925 09:27:17.635873 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:17.635920 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:17.638571 (+ 2651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:17.639261 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:17.639284 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:17.640042 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:17.640812 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:17.640837 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:17.642037 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:17.642045 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:17.643572 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:17.643583 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:17.644448 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:17.644455 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:17.644681 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:17.645389 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:17.645418 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:17.647450 (+ 2032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:17.650397 (+ 2947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:17.650405 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:17.650419 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:17.650822 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:17.651203 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:17.651209 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:17.651212 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:17.651432 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:17.651974 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:17.651981 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:17.652288 (+ 307us) client_negotiation.cc:769] Sending connection context
0925 09:27:17.652532 (+ 244us) client_negotiation.cc:240] Negotiation successful
0925 09:27:17.653056 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"thread_start_us":596,"threads_started":1}
WARNING: no policy specified for oryx/127.10.114.67@KRBTEST.COM; defaulting to no policy
I20260925 09:27:17.655514 17188 heartbeater.cc:347] Connected to a master server at 127.10.114.126:43419
I20260925 09:27:17.655766 17188 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.114.67@KRBTEST.COM" created.
I20260925 09:27:17.656548 17188 heartbeater.cc:511] Master 127.10.114.126:43419 requested a full tablet report, sending...
I20260925 09:27:17.657876 16865 ts_manager.cc:194] Registered new tserver with Master: 32afe2fd0d664cb588b6b4a2740e1106 (127.10.114.66:36841)
I20260925 09:27:17.658615 16865 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:60169
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.67.keytab.
Entry for principal oryx/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.67.keytab.
I20260925 09:27:17.679467 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43419
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.67.keytab
--principal=oryx/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:39685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:17.982609 17193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:17.982846 17193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:17.982995 17193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:17.989907 17193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:17.989995 17193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:17.990149 17193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:27:17.998811 17193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:39685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.67.keytab
--principal=oryx/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:18.000069 17193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:18.001281 17193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:18.013463 17198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:18.013660 17199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:18.013608 17201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:18.014722 17193 server_base.cc:1034] running on GCE node
I20260925 09:27:18.015363 17193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:18.016292 17193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:18.017519 17193 hybrid_clock.cc:648] HybridClock initialized: now 1790328438017484 us; error 51 us; skew 500 ppm
I20260925 09:27:18.017817 17193 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:18 dist-test-slave-9pj8 krb5kdc[16820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328438, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:18.020953 17193 init.cc:377] Logged in from keytab as oryx/127.10.114.67@KRBTEST.COM (short username oryx)
I20260925 09:27:18.022373 17193 webserver.cc:460] Webserver started at http://127.10.114.67:38423/ using document root <none> and password file <none>
I20260925 09:27:18.023032 17193 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:18.023212 17193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:18.023495 17193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:18.026301 17193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "2dbbfd760b8240e98b652fd145f7df82"
format_stamp: "Formatted at 2026-09-25 09:27:18 on dist-test-slave-9pj8"
server_key: "891fb3dc522934033684b7be895a296d"
server_key_iv: "09c0c5ed7738fa9e5caee2479fb7faf0"
server_key_version: "encryptionkey@0"
I20260925 09:27:18.027067 17193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "2dbbfd760b8240e98b652fd145f7df82"
format_stamp: "Formatted at 2026-09-25 09:27:18 on dist-test-slave-9pj8"
server_key: "891fb3dc522934033684b7be895a296d"
server_key_iv: "09c0c5ed7738fa9e5caee2479fb7faf0"
server_key_version: "encryptionkey@0"
I20260925 09:27:18.031764 17193 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260925 09:27:18.034926 17208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:18.036288 17193 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:27:18.036502 17193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "2dbbfd760b8240e98b652fd145f7df82"
format_stamp: "Formatted at 2026-09-25 09:27:18 on dist-test-slave-9pj8"
server_key: "891fb3dc522934033684b7be895a296d"
server_key_iv: "09c0c5ed7738fa9e5caee2479fb7faf0"
server_key_version: "encryptionkey@0"
I20260925 09:27:18.036676 17193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:18.049402 17193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:18.053371 17193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:18.053701 17193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:18.054525 17193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:18.055712 17193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:18.055838 17193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:18.055992 17193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:18.056077 17193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:18.087637 17193 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:46759
I20260925 09:27:18.087677 17320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:46759 every 8 connection(s)
I20260925 09:27:18.088806 17193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:27:18.095685 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 17193
I20260925 09:27:18.095888 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:27:18.096441 10697 external_mini_cluster.cc:1428] Setting key a33599f678031e291cae9d94a3700347
Sep 25 09:27:18 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328438, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM
I20260925 09:27:18.107612 17056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:18.091189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:46767 (local address 127.10.114.126:43419)
0925 09:27:18.091356 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:18.091360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:18.092611 (+ 1251us) server_negotiation.cc:407] Connection header received
0925 09:27:18.094371 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:18.094376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:18.094462 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:18.094567 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:18.096310 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.097119 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.099205 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.099618 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.099710 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:18.102989 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:18.102993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:18.102996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:18.103024 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:18.105385 (+ 2361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:18.105962 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:18.105966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:18.105968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:18.106068 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:18.106401 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:18.106403 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:18.106405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:18.106610 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:18.106722 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:18.107245 (+ 523us) server_negotiation.cc:299] Negotiation successful
0925 09:27:18.107394 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:27:18.108465 17323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:18.091638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.67:46767)
0925 09:27:18.092436 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:18.092470 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:18.094088 (+ 1618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:18.094807 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:18.094815 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:18.095285 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:18.096127 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:18.096151 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.097267 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.097272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:18.099039 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:18.099058 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.099846 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.099853 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:18.100100 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:18.100734 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:18.100755 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:18.102819 (+ 2064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:18.105500 (+ 2681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:18.105505 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:18.105515 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:18.105864 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:18.106158 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:18.106160 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:18.106162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:18.106316 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:18.106718 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:18.106723 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:18.106974 (+ 251us) client_negotiation.cc:769] Sending connection context
0925 09:27:18.107161 (+ 187us) client_negotiation.cc:240] Negotiation successful
0925 09:27:18.107491 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":342,"threads_started":1}
I20260925 09:27:18.109617 17321 heartbeater.cc:347] Connected to a master server at 127.10.114.126:43419
I20260925 09:27:18.109901 17321 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:18.110683 17321 heartbeater.cc:511] Master 127.10.114.126:43419 requested a full tablet report, sending...
I20260925 09:27:18.112022 16865 ts_manager.cc:194] Registered new tserver with Master: 2dbbfd760b8240e98b652fd145f7df82 (127.10.114.67:46759)
I20260925 09:27:18.112701 16865 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:46767
I20260925 09:27:18.123050 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.10.114.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.114.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.114.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.125.keytab.
Entry for principal oryx/127.10.114.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.125.keytab.
I20260925 09:27:18.168090 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.125:33507
--webserver_interface=127.10.114.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.125.keytab
--principal=oryx/127.10.114.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:39685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:43419,127.10.114.125:33507
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:27:18.190361 17054 heartbeater.cc:503] Master 127.10.114.126:43419 was elected leader, sending a full tablet report...
W20260925 09:27:18.474098 17326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:18.474385 17326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:18.474534 17326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:18.481279 17326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:27:18.481385 17326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:18.481458 17326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:18.481561 17326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:27:18.481617 17326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:27:18.490549 17326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:39685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:43419,127.10.114.125:33507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.125.keytab
--principal=oryx/127.10.114.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.125:33507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.114.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
W20260925 09:27:18.490883 17326 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.10.114.126:43419,127.10.114.125:33507'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260925 09:27:18.491838 17326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:18.493014 17326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:18.505774 17332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:18.505851 17334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:18.506405 17331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:18.507808 17326 server_base.cc:1034] running on GCE node
I20260925 09:27:18.508577 17326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:18.509778 17326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:18.511008 17326 hybrid_clock.cc:648] HybridClock initialized: now 1790328438510945 us; error 43 us; skew 500 ppm
I20260925 09:27:18.511312 17326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:18 dist-test-slave-9pj8 krb5kdc[16820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328438, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:18.514166 17326 init.cc:377] Logged in from keytab as oryx/127.10.114.125@KRBTEST.COM (short username oryx)
I20260925 09:27:18.515545 17326 webserver.cc:460] Webserver started at http://127.10.114.125:35097/ using document root <none> and password file <none>
I20260925 09:27:18.516264 17326 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:18.516389 17326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:18.516673 17326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:18.519439 17326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/data/instance:
uuid: "0ad69ec73960431382f3b6085c0b0360"
format_stamp: "Formatted at 2026-09-25 09:27:18 on dist-test-slave-9pj8"
server_key: "020fecb1140f53bb24d79d75dc9c3fcc"
server_key_iv: "79fe12a5f4359c26729274270a2c8cf9"
server_key_version: "encryptionkey@0"
I20260925 09:27:18.520187 17326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/wal/instance:
uuid: "0ad69ec73960431382f3b6085c0b0360"
format_stamp: "Formatted at 2026-09-25 09:27:18 on dist-test-slave-9pj8"
server_key: "020fecb1140f53bb24d79d75dc9c3fcc"
server_key_iv: "79fe12a5f4359c26729274270a2c8cf9"
server_key_version: "encryptionkey@0"
I20260925 09:27:18.525074 17326 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260925 09:27:18.528247 17341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:18.529418 17326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 09:27:18.529593 17326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/wal
uuid: "0ad69ec73960431382f3b6085c0b0360"
format_stamp: "Formatted at 2026-09-25 09:27:18 on dist-test-slave-9pj8"
server_key: "020fecb1140f53bb24d79d75dc9c3fcc"
server_key_iv: "79fe12a5f4359c26729274270a2c8cf9"
server_key_version: "encryptionkey@0"
I20260925 09:27:18.529789 17326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:18.558915 17326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:18.562697 17326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:18.562937 17326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:18.580166 17326 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.125:33507
I20260925 09:27:18.580183 17392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.125:33507 every 8 connection(s)
I20260925 09:27:18.581451 17326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/data/info.pb
I20260925 09:27:18.583967 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 17326
I20260925 09:27:18.585618 17393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:18.593362 17393 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:27:18 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.114.125@KRBTEST.COM
Sep 25 09:27:18 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328438, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.125@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM
I20260925 09:27:18.607501 17396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:18.586243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34232 (local address 127.10.114.125:33507)
0925 09:27:18.586952 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:18.586963 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:18.587001 (+ 38us) server_negotiation.cc:407] Connection header received
0925 09:27:18.588470 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:18.588490 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:18.588840 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:18.589203 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:18.590559 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.591917 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.593151 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.593816 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.594026 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:18.599029 (+ 5003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:18.599042 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:18.599062 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:18.599114 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:18.603045 (+ 3931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:18.603762 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:18.603792 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:18.603800 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:18.603991 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:18.604721 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:18.604727 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:18.604730 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:18.605357 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:18.605626 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:18.606059 (+ 433us) server_negotiation.cc:299] Negotiation successful
0925 09:27:18.606502 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":329,"threads_started":1}
I20260925 09:27:18.612457 17056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:18.596027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52546 (local address 127.10.114.126:43419)
0925 09:27:18.596400 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:18.596410 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:18.597593 (+ 1183us) server_negotiation.cc:407] Connection header received
0925 09:27:18.597903 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:18.597910 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:18.598003 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:18.598106 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:18.599363 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.600505 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.602695 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.603384 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.603504 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:18.607038 (+ 3534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:18.607043 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:18.607046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:18.607077 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:18.609689 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:18.610486 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:18.610495 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:18.610499 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:18.610624 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:18.611002 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:18.611007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:18.611010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:18.611330 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:18.611478 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:18.611918 (+ 440us) server_negotiation.cc:299] Negotiation successful
0925 09:27:18.612113 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211}
I20260925 09:27:18.612505 17397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:18.596074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.0.0.1:52546)
0925 09:27:18.597467 (+ 1393us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:18.597508 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:18.597734 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:18.598158 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:18.598168 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:18.598766 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:18.599186 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:18.599204 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.600715 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.600720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:18.602526 (+ 1806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:18.602541 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.603574 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.603581 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:18.603844 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:18.604464 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:18.604490 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:18.606830 (+ 2340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:18.609823 (+ 2993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:18.609832 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:18.609839 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:18.610368 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:18.610739 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:18.610744 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:18.610747 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:18.610876 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:18.611464 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:18.611472 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:18.611631 (+ 159us) client_negotiation.cc:769] Sending connection context
0925 09:27:18.611829 (+ 198us) client_negotiation.cc:240] Negotiation successful
0925 09:27:18.612111 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1165,"thread_start_us":1014,"threads_started":1}
I20260925 09:27:18.613979 17393 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:27:18 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328438, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.125@KRBTEST.COM for oryx/127.10.114.125@KRBTEST.COM
I20260925 09:27:18.625300 17397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:18.614667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33507 (local address 127.0.0.1:34234)
0925 09:27:18.614864 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:18.614879 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:18.614949 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:18.615191 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:18.615194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:18.615462 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:18.615577 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:18.615584 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.616552 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.616557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:18.617406 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:18.617416 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.618098 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.618102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:18.618222 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:18.618662 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:18.618677 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:18.620671 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:18.623427 (+ 2756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:18.623430 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:18.623433 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:18.623862 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:18.624151 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:18.624153 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:18.624155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:18.624280 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:18.624689 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:18.624692 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:18.624793 (+ 101us) client_negotiation.cc:769] Sending connection context
0925 09:27:18.624891 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 09:27:18.625057 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260925 09:27:18.625527 17396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:18.614754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34234 (local address 127.10.114.125:33507)
0925 09:27:18.614919 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:18.614923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:18.614938 (+ 15us) server_negotiation.cc:407] Connection header received
0925 09:27:18.615037 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:18.615040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:18.615088 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:18.615161 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:18.615686 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.616414 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.617545 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.617949 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.618045 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:18.620832 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:18.620836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:18.620839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:18.620864 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:18.623306 (+ 2442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:18.623954 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:18.623957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:18.623959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:18.624062 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:18.624363 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:18.624366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:18.624367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:18.624584 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:18.624709 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:18.624943 (+ 234us) server_negotiation.cc:299] Negotiation successful
0925 09:27:18.625161 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":71}
I20260925 09:27:18.629869 17393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360: Bootstrap starting.
I20260925 09:27:18.632833 17393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:18.633970 17393 log.cc:824] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:18.636459 17393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360: No bootstrap required, opened a new log
I20260925 09:27:18.643276 17393 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } }
I20260925 09:27:18.643568 17393 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:18.643674 17393 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad69ec73960431382f3b6085c0b0360, State: Initialized, Role: FOLLOWER
I20260925 09:27:18.644200 17393 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } }
I20260925 09:27:18.644959 17399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } } }
I20260925 09:27:18.645278 17399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:27:18.645917 17393 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: configured and running, proceeding with master startup.
W20260925 09:27:18.654683 17410 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:27:18.654829 17410 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:27:18.656634 17326 master_runner.cc:428] Detected that this master 0ad69ec73960431382f3b6085c0b0360 is joining an existing cluster
I20260925 09:27:18.656718 17326 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.114.125:33507
I20260925 09:27:18.661180 17188 heartbeater.cc:503] Master 127.10.114.126:43419 was elected leader, sending a full tablet report...
I20260925 09:27:18.683897 17396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:18.660477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34246 (local address 127.10.114.125:33507)
0925 09:27:18.660636 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:18.660640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:18.668911 (+ 8271us) server_negotiation.cc:407] Connection header received
0925 09:27:18.669069 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:18.669073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:18.669134 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:18.669187 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:18.670659 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.671753 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.674036 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.674767 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.674900 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:18.677015 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:18.677022 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:18.677026 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:18.677055 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:18.680854 (+ 3799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:18.681676 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:18.681684 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:18.681687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:18.681814 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:18.682271 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:18.682277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:18.682280 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:18.682618 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:18.682767 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:18.683264 (+ 497us) server_negotiation.cc:299] Negotiation successful
0925 09:27:18.683465 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 09:27:18.684823 17416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:18.660195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33507 (local address 127.0.0.1:34246)
0925 09:27:18.668842 (+ 8647us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:18.668857 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:18.668974 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:18.669397 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:18.669403 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:18.670065 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:18.670478 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:18.670487 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.671970 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.671976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:18.673889 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:18.673902 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.674931 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.674953 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:18.675064 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:18.675664 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:18.675680 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:18.676827 (+ 1147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:18.681047 (+ 4220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:18.681054 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:18.681061 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:18.681560 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:18.681939 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:18.681945 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:18.681949 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:18.682161 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:18.682864 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:18.682868 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:18.683005 (+ 137us) client_negotiation.cc:769] Sending connection context
0925 09:27:18.684135 (+ 1130us) client_negotiation.cc:240] Negotiation successful
0925 09:27:18.684378 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8506,"mutex_wait_us":171,"thread_start_us":8325,"threads_started":1}
WARNING: no policy specified for oryx/127.10.114.124@KRBTEST.COM; defaulting to no policy
I20260925 09:27:18.686029 17056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:18.660027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52554 (local address 127.10.114.126:43419)
0925 09:27:18.660199 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:18.660216 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:18.668566 (+ 8350us) server_negotiation.cc:407] Connection header received
0925 09:27:18.668833 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:18.668838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:18.668921 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:18.668988 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:18.670664 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.671736 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.675132 (+ 3396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.675947 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.676083 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:18.678711 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:18.678717 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:18.678721 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:18.678760 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:18.682623 (+ 3863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:18.683335 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:18.683340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:18.683343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:18.683479 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:18.683925 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:18.683931 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:18.683934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:18.684562 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:18.684669 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:18.685499 (+ 830us) server_negotiation.cc:299] Negotiation successful
0925 09:27:18.685716 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 09:27:18.686945 17417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:18.660162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.0.0.1:52554)
0925 09:27:18.668448 (+ 8286us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:18.668479 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:18.668681 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:18.669170 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:18.669176 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:18.669978 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:18.670443 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:18.670461 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.671947 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.671953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:18.673889 (+ 1936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:18.673902 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.676149 (+ 2247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.676155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:18.676374 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:18.676991 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:18.677012 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:18.678485 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:18.682768 (+ 4283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:18.682772 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:18.682777 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:18.683202 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:18.683614 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:18.683619 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:18.683623 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:18.683762 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:18.684755 (+ 993us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:18.684761 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:18.685130 (+ 369us) client_negotiation.cc:769] Sending connection context
0925 09:27:18.686264 (+ 1134us) client_negotiation.cc:240] Negotiation successful
0925 09:27:18.686554 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8139,"mutex_wait_us":120,"thread_start_us":7986,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.114.124@KRBTEST.COM" created.
Sep 25 09:27:18 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328436, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.126@KRBTEST.COM for oryx/127.10.114.125@KRBTEST.COM
I20260925 09:27:18.706983 17396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:18.693197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34262 (local address 127.10.114.125:33507)
0925 09:27:18.693354 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:18.693360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:18.693941 (+ 581us) server_negotiation.cc:407] Connection header received
0925 09:27:18.694334 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:18.694342 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:18.694429 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:18.694537 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:18.695505 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.696346 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.697575 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.698131 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.698264 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:18.701684 (+ 3420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:18.701689 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:18.701692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:18.701724 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:18.704461 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:18.705106 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:18.705114 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:18.705117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:18.705257 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:18.705604 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:18.705610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:18.705614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:18.705844 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:18.705989 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:18.706432 (+ 443us) server_negotiation.cc:299] Negotiation successful
0925 09:27:18.706629 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 09:27:18.707047 17419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:18.693232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33507 (local address 127.0.0.1:34262)
0925 09:27:18.693819 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:18.693858 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:18.694102 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:18.694587 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:18.694594 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:18.695162 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:18.695354 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:18.695362 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.696472 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.696478 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:18.697447 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:18.697457 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.698467 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.698472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:18.698682 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:18.699304 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:18.699320 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:18.701509 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:18.704584 (+ 3075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:18.704594 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:18.704601 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:18.704993 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:18.705366 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:18.705371 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:18.705374 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:18.705502 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:18.705973 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:18.705980 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:18.706138 (+ 158us) client_negotiation.cc:769] Sending connection context
0925 09:27:18.706322 (+ 184us) client_negotiation.cc:240] Negotiation successful
0925 09:27:18.706623 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":231,"threads_started":1}
I20260925 09:27:18.708972 16865 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.114.125:33507: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0ad69ec73960431382f3b6085c0b0360"
member_type: NON_VOTER
last_known_addr {
host: "127.10.114.125"
port: 33507
}
attrs {
promote: true
}
}
dest_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04"
cas_config_opid_index: -1
I20260925 09:27:18.709894 16865 consensus_queue.cc:237] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: NON_VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: true } }
I20260925 09:27:18.712811 17422 raft_consensus.cc:2932] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0ad69ec73960431382f3b6085c0b0360 (127.10.114.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: NON_VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: true } } }
I20260925 09:27:18.713667 17368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:18.717624 17368 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 1 FOLLOWER]: Refusing update from remote peer de3ad4b7da3040669e2c44a8dfe9cf04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.114.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.124.keytab.
Entry for principal oryx/127.10.114.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.124.keytab.
I20260925 09:27:18.721719 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.124:33261
--webserver_interface=127.10.114.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.124.keytab
--principal=oryx/127.10.114.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:39685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:43419,127.10.114.125:33507,127.10.114.124:33261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:27:18.723752 17423 consensus_queue.cc:1035] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: NON_VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:27:18.725366 17422 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.114.125:33507
I20260925 09:27:18.726044 17326 master.cc:537] Master@127.10.114.125:33507 shutting down...
I20260925 09:27:18.727031 17424 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: NON_VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: true } } }
I20260925 09:27:18.727360 17424 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:18.755859 17368 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0ad69ec73960431382f3b6085c0b0360 (127.10.114.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: NON_VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: true } } }
I20260925 09:27:18.779735 17433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: NON_VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: true } } }
I20260925 09:27:18.780202 17433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: This master's current role is: LEARNER
I20260925 09:27:18.780282 17399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de3ad4b7da3040669e2c44a8dfe9cf04. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: NON_VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: true } } }
I20260925 09:27:18.780500 17399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: This master's current role is: LEARNER
I20260925 09:27:18.780579 17429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: NON_VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: true } } }
I20260925 09:27:18.780921 17429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: This master's current role is: LEARNER
I20260925 09:27:18.783661 17326 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 1 LEARNER]: Raft consensus shutting down.
I20260925 09:27:18.784127 17326 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 1 LEARNER]: Raft consensus is shut down!
I20260925 09:27:18.784296 17326 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360: stopping tablet replica
I20260925 09:27:18.792279 17326 master.cc:552] Master@127.10.114.125:33507 shutdown complete.
I20260925 09:27:18.792438 17326 master_runner.cc:305] Clearing existing system tablet
I20260925 09:27:18.793000 17326 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:18.793149 17326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:18.796295 17326 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260925 09:27:18.797928 17435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:18.798231 17326 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:18.798635 17326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 09:27:18.798779 17326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/wal
uuid: "0ad69ec73960431382f3b6085c0b0360"
format_stamp: "Formatted at 2026-09-25 09:27:18 on dist-test-slave-9pj8"
server_key: "020fecb1140f53bb24d79d75dc9c3fcc"
server_key_iv: "79fe12a5f4359c26729274270a2c8cf9"
server_key_version: "encryptionkey@0"
I20260925 09:27:18.803069 17326 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:18.810767 17326 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 09:27:18.810987 17326 log.cc:1196] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260925 09:27:18.811568 17326 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360: Deleting consensus metadata
I20260925 09:27:18.811906 17326 master_runner.cc:315] Copying system tablet from 127.10.114.126:43419
I20260925 09:27:18.814575 17326 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360: tablet copy: Beginning tablet copy session from remote peer at address 127.10.114.126:43419
I20260925 09:27:18.837780 17056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:18.815453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52568 (local address 127.10.114.126:43419)
0925 09:27:18.815623 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:18.815629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:18.824049 (+ 8420us) server_negotiation.cc:407] Connection header received
0925 09:27:18.824309 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:18.824316 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:18.824413 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:18.824523 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:18.825569 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.826684 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.828592 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.829219 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.829340 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:18.831501 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:18.831507 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:18.831512 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:18.831549 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:18.834857 (+ 3308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:18.835639 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:18.835644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:18.835648 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:18.835811 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:18.836260 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:18.836265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:18.836268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:18.836629 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:18.836769 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:18.837233 (+ 464us) server_negotiation.cc:299] Negotiation successful
0925 09:27:18.837448 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 09:27:18.838178 17441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:18.815385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.0.0.1:52568)
0925 09:27:18.823864 (+ 8479us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:18.823896 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:18.824155 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:18.824568 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:18.824587 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:18.825111 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:18.825409 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:18.825421 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.826845 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.826863 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:18.828420 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:18.828434 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:18.829377 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:18.829382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:18.829554 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:18.830235 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:18.830277 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:18.831300 (+ 1023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:18.835022 (+ 3722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:18.835027 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:18.835030 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:18.835526 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:18.835923 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:18.835928 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:18.835932 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:18.836121 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:18.836754 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:18.836759 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:18.836943 (+ 184us) client_negotiation.cc:769] Sending connection context
0925 09:27:18.837124 (+ 181us) client_negotiation.cc:240] Negotiation successful
0925 09:27:18.837410 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8285,"thread_start_us":8140,"threads_started":1}
I20260925 09:27:18.839015 16885 tablet_copy_service.cc:140] P de3ad4b7da3040669e2c44a8dfe9cf04: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0ad69ec73960431382f3b6085c0b0360 ({username='oryx', principal='oryx/127.10.114.125@KRBTEST.COM'} at 127.0.0.1:52568)
I20260925 09:27:18.839290 16885 tablet_copy_service.cc:161] P de3ad4b7da3040669e2c44a8dfe9cf04: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0ad69ec73960431382f3b6085c0b0360 at {username='oryx', principal='oryx/127.10.114.125@KRBTEST.COM'} at 127.0.0.1:52568: session id = 0ad69ec73960431382f3b6085c0b0360-00000000000000000000000000000000
I20260925 09:27:18.843722 16885 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 09:27:18.845800 17326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 09:27:18.852360 17326 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360: tablet copy: Starting download of 0 data blocks...
I20260925 09:27:18.852634 17326 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360: tablet copy: Starting download of 1 WAL segments...
I20260925 09:27:18.855028 17326 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 09:27:18.859444 16885 tablet_copy_service.cc:342] P de3ad4b7da3040669e2c44a8dfe9cf04: Request end of tablet copy session 0ad69ec73960431382f3b6085c0b0360-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.114.125@KRBTEST.COM'} at 127.0.0.1:52568
I20260925 09:27:18.859625 16885 tablet_copy_service.cc:433] P de3ad4b7da3040669e2c44a8dfe9cf04: ending tablet copy session 0ad69ec73960431382f3b6085c0b0360-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0ad69ec73960431382f3b6085c0b0360
I20260925 09:27:18.871246 17326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:18.873978 17442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:18.874717 17443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:18.882858 17326 server_base.cc:1034] running on GCE node
W20260925 09:27:18.882863 17445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:18.883476 17326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:18.884089 17326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:18.885291 17326 hybrid_clock.cc:648] HybridClock initialized: now 1790328438885219 us; error 48 us; skew 500 ppm
I20260925 09:27:18.885509 17326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:18 dist-test-slave-9pj8 krb5kdc[16820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328438, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:18.887604 17326 init.cc:377] Logged in from keytab as oryx/127.10.114.125@KRBTEST.COM (short username oryx)
I20260925 09:27:18.892786 17326 webserver.cc:460] Webserver started at http://127.10.114.125:33137/ using document root <none> and password file <none>
I20260925 09:27:18.893158 17326 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:18.893244 17326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:18.896289 17326 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260925 09:27:18.898272 17452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:18.898715 17326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 09:27:18.899154 17326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/wal
uuid: "0ad69ec73960431382f3b6085c0b0360"
format_stamp: "Formatted at 2026-09-25 09:27:18 on dist-test-slave-9pj8"
server_key: "020fecb1140f53bb24d79d75dc9c3fcc"
server_key_iv: "79fe12a5f4359c26729274270a2c8cf9"
server_key_version: "encryptionkey@0"
I20260925 09:27:18.899395 17326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:18.933001 17326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:18.937161 17326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:18.977783 17326 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.125:33507
I20260925 09:27:18.977828 17503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.125:33507 every 8 connection(s)
I20260925 09:27:18.978741 17326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-1/data/info.pb
I20260925 09:27:18.984498 17504 sys_catalog.cc:263] Verifying existing consensus state
I20260925 09:27:18.988430 17504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360: Bootstrap starting.
I20260925 09:27:19.002569 17504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 09:27:19.003019 17504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360: Bootstrap complete.
I20260925 09:27:19.004145 17504 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: NON_VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: true } }
I20260925 09:27:19.004443 17504 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0ad69ec73960431382f3b6085c0b0360, State: Initialized, Role: LEARNER
I20260925 09:27:19.004711 17504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: NON_VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: true } }
I20260925 09:27:19.005517 17507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: NON_VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: true } } }
I20260925 09:27:19.005891 17504 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:27:19.005926 17507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: This master's current role is: LEARNER
I20260925 09:27:19.014565 17518 catalog_manager.cc:1259] Loaded cluster ID: 5344fdedc2704f5782bd1313986aaf4e
I20260925 09:27:19.014662 17518 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360: loading cluster ID for follower catalog manager: success
I20260925 09:27:19.018177 17518 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360: acquiring CA information for follower catalog manager: success
I20260925 09:27:19.020576 17518 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 09:27:19.102761 17427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:19.103053 17427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:19.103199 17427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:19.110893 17427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:27:19.111032 17427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:19.111138 17427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:19.111229 17427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:27:19.111325 17427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:27:19.115046 17321 heartbeater.cc:503] Master 127.10.114.126:43419 was elected leader, sending a full tablet report...
I20260925 09:27:19.121645 17427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:39685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:43419,127.10.114.125:33507,127.10.114.124:33261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.124.keytab
--principal=oryx/127.10.114.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.124:33261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.114.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:19.122753 17427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:19.124045 17427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:19.135496 17524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:19.135804 17523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:19.136582 17526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:19.136793 17427 server_base.cc:1034] running on GCE node
I20260925 09:27:19.137521 17427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:19.138727 17427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:19.139937 17427 hybrid_clock.cc:648] HybridClock initialized: now 1790328439139863 us; error 60 us; skew 500 ppm
I20260925 09:27:19.140256 17427 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:19 dist-test-slave-9pj8 krb5kdc[16820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328439, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:19.143460 17427 init.cc:377] Logged in from keytab as oryx/127.10.114.124@KRBTEST.COM (short username oryx)
I20260925 09:27:19.145123 17427 webserver.cc:460] Webserver started at http://127.10.114.124:46321/ using document root <none> and password file <none>
I20260925 09:27:19.145757 17427 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:19.145895 17427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:19.146191 17427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:19.149115 17427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/data/instance:
uuid: "fe4bf2501e9144b79c313dc56a923f42"
format_stamp: "Formatted at 2026-09-25 09:27:19 on dist-test-slave-9pj8"
server_key: "ebe5bfe30a9633ecc9c632940b83e9d9"
server_key_iv: "30511973841ad96d1fad98abf9e6cd39"
server_key_version: "encryptionkey@0"
I20260925 09:27:19.149855 17427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/wal/instance:
uuid: "fe4bf2501e9144b79c313dc56a923f42"
format_stamp: "Formatted at 2026-09-25 09:27:19 on dist-test-slave-9pj8"
server_key: "ebe5bfe30a9633ecc9c632940b83e9d9"
server_key_iv: "30511973841ad96d1fad98abf9e6cd39"
server_key_version: "encryptionkey@0"
I20260925 09:27:19.155145 17427 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:27:19.158684 17533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:19.160115 17427 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:27:19.160276 17427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/wal
uuid: "fe4bf2501e9144b79c313dc56a923f42"
format_stamp: "Formatted at 2026-09-25 09:27:19 on dist-test-slave-9pj8"
server_key: "ebe5bfe30a9633ecc9c632940b83e9d9"
server_key_iv: "30511973841ad96d1fad98abf9e6cd39"
server_key_version: "encryptionkey@0"
I20260925 09:27:19.160437 17427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:19.183403 17427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:19.187417 17427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:19.187716 17427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:19.207988 17422 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: NON_VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: true } } }
I20260925 09:27:19.208305 17422 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:19.210000 17427 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.124:33261
I20260925 09:27:19.210074 17586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.124:33261 every 8 connection(s)
I20260925 09:27:19.211434 17427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/data/info.pb
I20260925 09:27:19.215651 17587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:19.218112 17585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.208481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34274 (local address 127.10.114.125:33507)
0925 09:27:19.212167 (+ 3686us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:19.212171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:19.212186 (+ 15us) server_negotiation.cc:407] Connection header received
0925 09:27:19.212241 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:19.212244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:19.212332 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:19.212917 (+ 585us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:19.213065 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.213992 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.216429 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.217190 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.217335 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.217447 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:19.217781 (+ 334us) server_negotiation.cc:299] Negotiation successful
0925 09:27:19.217872 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3571,"thread_start_us":3439,"threads_started":1}
I20260925 09:27:19.218269 17583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.208328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33507 (local address 127.0.0.1:34274)
0925 09:27:19.208786 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:19.208812 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:19.208973 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:19.212554 (+ 3581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:19.212560 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:19.212591 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:19.212805 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.212813 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.214159 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.214165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.216275 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.216291 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.217361 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.217367 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.217586 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.217647 (+ 61us) client_negotiation.cc:769] Sending connection context
0925 09:27:19.217766 (+ 119us) client_negotiation.cc:240] Negotiation successful
0925 09:27:19.217912 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":239,"threads_started":1}
I20260925 09:27:19.219334 17507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de3ad4b7da3040669e2c44a8dfe9cf04. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: NON_VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: true } } }
I20260925 09:27:19.219594 17507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: This master's current role is: LEARNER
I20260925 09:27:19.220077 17422 raft_consensus.cc:1050] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04: attempting to promote NON_VOTER 0ad69ec73960431382f3b6085c0b0360 to VOTER
I20260925 09:27:19.220376 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 17427
I20260925 09:27:19.221400 17422 consensus_queue.cc:237] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } }
I20260925 09:27:19.223927 17479 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 1 LEARNER]: Refusing update from remote peer de3ad4b7da3040669e2c44a8dfe9cf04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 09:27:19.224443 17424 consensus_queue.cc:1035] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 09:27:19.223726 17587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:27:19.227035 17424 raft_consensus.cc:2932] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 0ad69ec73960431382f3b6085c0b0360 (127.10.114.125) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } }
I20260925 09:27:19.228739 17479 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 0ad69ec73960431382f3b6085c0b0360 (127.10.114.125) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } }
I20260925 09:27:19.231325 17422 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } }
I20260925 09:27:19.231523 17422 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:19.231921 17422 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } }
I20260925 09:27:19.232120 17422 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:19.232852 17507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } }
I20260925 09:27:19.233052 17507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: This master's current role is: FOLLOWER
Sep 25 09:27:19 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.114.124@KRBTEST.COM
Sep 25 09:27:19 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328439, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.124@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM
I20260925 09:27:19.245872 17056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.228368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52584 (local address 127.10.114.126:43419)
0925 09:27:19.228625 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:19.228634 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:19.229405 (+ 771us) server_negotiation.cc:407] Connection header received
0925 09:27:19.229673 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:19.229681 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:19.229766 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:19.229853 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:19.231524 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.232679 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.234854 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.235531 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.235686 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.239616 (+ 3930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:19.239620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:19.239623 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:19.239652 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:19.242982 (+ 3330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.243737 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.243741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.243743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.243929 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.244324 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.244328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.244330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.244587 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:19.244686 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:19.245367 (+ 681us) server_negotiation.cc:299] Negotiation successful
0925 09:27:19.245587 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 09:27:19.246858 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.222901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43034 (local address 127.10.114.124:33261)
0925 09:27:19.223761 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:19.223808 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:19.223857 (+ 49us) server_negotiation.cc:407] Connection header received
0925 09:27:19.225958 (+ 2101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:19.225987 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:19.226465 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:19.226876 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:19.229826 (+ 2950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.231914 (+ 2088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.233649 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.234626 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.234783 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.239104 (+ 4321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:19.239114 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:19.239128 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:19.239195 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:19.242792 (+ 3597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.243509 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.243518 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.243523 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.243717 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.244216 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.244221 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.244224 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.244769 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:19.245043 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:19.245323 (+ 280us) server_negotiation.cc:299] Negotiation successful
0925 09:27:19.245712 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":610,"thread_start_us":370,"threads_started":1}
I20260925 09:27:19.247565 17592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.228014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.0.0.1:52584)
0925 09:27:19.229158 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:19.229317 (+ 159us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:19.229518 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:19.230065 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:19.230071 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:19.230968 (+ 897us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:19.231349 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.231366 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.232852 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.232857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.234576 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.234591 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.235832 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.235841 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.236011 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.236646 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:19.236670 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:19.239385 (+ 2715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:19.243123 (+ 3738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:19.243132 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:19.243141 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:19.243617 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:19.244040 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:19.244043 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:19.244045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:19.244215 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:19.244734 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:19.244742 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:19.244945 (+ 203us) client_negotiation.cc:769] Sending connection context
0925 09:27:19.245253 (+ 308us) client_negotiation.cc:240] Negotiation successful
0925 09:27:19.247293 (+ 2040us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":923,"mutex_wait_us":1707,"thread_start_us":792,"threads_started":1}
I20260925 09:27:19.248837 17587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:27:19 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328439, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.124@KRBTEST.COM for oryx/127.10.114.125@KRBTEST.COM
I20260925 09:27:19.260301 17592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.249438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33507 (local address 127.0.0.1:34276)
0925 09:27:19.249595 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:19.249614 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:19.249704 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:19.250016 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:19.250021 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:19.250339 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:19.250454 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.250462 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.251446 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.251451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.252452 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.252462 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.253144 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.253149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.253354 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.253871 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:19.253887 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:19.255930 (+ 2043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:19.258475 (+ 2545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:19.258478 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:19.258481 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:19.258813 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:19.259125 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:19.259128 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:19.259130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:19.259243 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:19.259636 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:19.259639 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:19.259747 (+ 108us) client_negotiation.cc:769] Sending connection context
0925 09:27:19.259919 (+ 172us) client_negotiation.cc:240] Negotiation successful
0925 09:27:19.260090 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 09:27:19.260353 17585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.249580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34276 (local address 127.10.114.125:33507)
0925 09:27:19.249768 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:19.249774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:19.249792 (+ 18us) server_negotiation.cc:407] Connection header received
0925 09:27:19.249845 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:19.249848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:19.249901 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:19.250000 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:19.250594 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.251313 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.252597 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.253009 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.253137 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.256087 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:19.256091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:19.256095 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:19.256123 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:19.258366 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.258904 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.258907 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.258909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.259037 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.259327 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.259330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.259332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.259547 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:19.259648 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:19.259986 (+ 338us) server_negotiation.cc:299] Negotiation successful
0925 09:27:19.260138 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 09:27:19.261344 17587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:27:19 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328439, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.124@KRBTEST.COM for oryx/127.10.114.124@KRBTEST.COM
I20260925 09:27:19.273299 17592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.262028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:33261 (local address 127.0.0.1:43042)
0925 09:27:19.262147 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:19.262161 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:19.262299 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:19.262564 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:19.262568 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:19.262889 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:19.263008 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.263015 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.264404 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.264412 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.265284 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.265294 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.265911 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.265916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.266130 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.266557 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:19.266571 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:19.268345 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:19.271028 (+ 2683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:19.271035 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:19.271040 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:19.271355 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:19.271682 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:19.271696 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:19.271700 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:19.271890 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:19.272370 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:19.272374 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:19.272500 (+ 126us) client_negotiation.cc:769] Sending connection context
0925 09:27:19.272638 (+ 138us) client_negotiation.cc:240] Negotiation successful
0925 09:27:19.272896 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":4}
I20260925 09:27:19.273337 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.262159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43042 (local address 127.10.114.124:33261)
0925 09:27:19.262301 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:19.262306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:19.262324 (+ 18us) server_negotiation.cc:407] Connection header received
0925 09:27:19.262395 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:19.262400 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:19.262456 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:19.262551 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:19.263127 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.264292 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.265407 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.265784 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.265905 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.268503 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:19.268508 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:19.268513 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:19.268620 (+ 107us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:19.270893 (+ 2273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.271463 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.271471 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.271474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.271583 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.271994 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.271998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.272002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.272239 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:19.272392 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:19.272706 (+ 314us) server_negotiation.cc:299] Negotiation successful
0925 09:27:19.273020 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":178,"server-negotiator.queue_time_us":42}
I20260925 09:27:19.277732 17587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: Bootstrap starting.
I20260925 09:27:19.280730 17587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:19.282153 17587 log.cc:824] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:19.284926 17587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: No bootstrap required, opened a new log
I20260925 09:27:19.291498 17587 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } }
I20260925 09:27:19.291815 17587 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:19.291944 17587 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe4bf2501e9144b79c313dc56a923f42, State: Initialized, Role: FOLLOWER
I20260925 09:27:19.292440 17587 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } }
I20260925 09:27:19.293187 17594 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } } }
I20260925 09:27:19.293453 17594 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:27:19.294128 17587 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:27:19.301741 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 16926
W20260925 09:27:19.303921 17605 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:27:19.304080 17605 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:27:19.307819 17427 master_runner.cc:428] Detected that this master fe4bf2501e9144b79c313dc56a923f42 is joining an existing cluster
I20260925 09:27:19.307934 17427 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.114.124:33261
Sep 25 09:27:19 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.114.65@KRBTEST.COM
I20260925 09:27:19.324026 17607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.304051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40416 (local address 127.10.114.65:38265)
0925 09:27:19.304671 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:19.304679 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:19.304704 (+ 25us) server_negotiation.cc:407] Connection header received
0925 09:27:19.304765 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:19.304771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:19.305018 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:19.305204 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:19.306679 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.307743 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.309276 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.309863 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.309935 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.315710 (+ 5775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:19.315722 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:19.315736 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:19.315807 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:19.319900 (+ 4093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.320818 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.320826 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.320830 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.320991 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.321479 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.321484 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.321489 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.321893 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:19.323200 (+ 1307us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:19.323351 (+ 151us) server_negotiation.cc:299] Negotiation successful
0925 09:27:19.323601 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":311,"threads_started":1}
I20260925 09:27:19.325495 17049 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:19.361690 17585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.313476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34280 (local address 127.10.114.125:33507)
0925 09:27:19.313645 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:19.313652 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:19.327354 (+ 13702us) server_negotiation.cc:407] Connection header received
0925 09:27:19.328670 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:19.328678 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:19.328785 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:19.328876 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:19.335097 (+ 6221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.336807 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.345656 (+ 8849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.346366 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.346478 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.352769 (+ 6291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:19.352777 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:19.352790 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:19.352837 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:19.357265 (+ 4428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.358899 (+ 1634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.358909 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.358913 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.359131 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.359676 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.359684 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.359689 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.360095 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:19.360291 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:19.360984 (+ 693us) server_negotiation.cc:299] Negotiation successful
0925 09:27:19.361238 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:27:19.365049 17056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.313855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52594 (local address 127.10.114.126:43419)
0925 09:27:19.314054 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:19.314061 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:19.345419 (+ 31358us) server_negotiation.cc:407] Connection header received
0925 09:27:19.345597 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:19.345606 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:19.345702 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:19.345805 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:19.347295 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.348476 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.350692 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.351440 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.351546 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.355007 (+ 3461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:19.355015 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:19.355036 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:19.355079 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:19.358660 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.362663 (+ 4003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.362682 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.362686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.362860 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.363302 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.363307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.363310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.363645 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:19.363811 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:19.364424 (+ 613us) server_negotiation.cc:299] Negotiation successful
0925 09:27:19.364634 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 09:27:19.365406 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.315935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43058 (local address 127.10.114.124:33261)
0925 09:27:19.316079 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:19.316086 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:19.332713 (+ 16627us) server_negotiation.cc:407] Connection header received
0925 09:27:19.333026 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:19.333033 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:19.333112 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:19.333203 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:19.340156 (+ 6953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.341580 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.348024 (+ 6444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.348809 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.348920 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.358151 (+ 9231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:19.358158 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:19.358164 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:19.358213 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:19.361867 (+ 3654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.362875 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.362880 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.362884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.363037 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.363445 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.363451 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.363454 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.363909 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:19.364035 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:19.364497 (+ 462us) server_negotiation.cc:299] Negotiation successful
0925 09:27:19.364954 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":231,"server-negotiator.queue_time_us":30}
I20260925 09:27:19.366876 17613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.319766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.0.0.1:52594)
0925 09:27:19.345311 (+ 25545us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:19.345331 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:19.345466 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:19.345970 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:19.345976 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:19.346635 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:19.347117 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.347129 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.348706 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.348712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.350489 (+ 1777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.350504 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.351890 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.351898 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.352136 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.353057 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:19.353080 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:19.354731 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:19.362046 (+ 7315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:19.362053 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:19.362058 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:19.362501 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:19.363016 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:19.363020 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:19.363024 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:19.363170 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:19.363905 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:19.363909 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:19.364135 (+ 226us) client_negotiation.cc:769] Sending connection context
0925 09:27:19.365326 (+ 1191us) client_negotiation.cc:240] Negotiation successful
0925 09:27:19.365657 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25322,"mutex_wait_us":26,"thread_start_us":7172,"threads_started":1}
I20260925 09:27:19.367409 17614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.313459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33507 (local address 127.0.0.1:34280)
0925 09:27:19.327211 (+ 13752us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:19.327242 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:19.328296 (+ 1054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:19.329223 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:19.329231 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:19.330156 (+ 925us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:19.330641 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.330659 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.337263 (+ 6604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.337271 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.339976 (+ 2705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.344680 (+ 4704us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.346798 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.346811 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.347046 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.349861 (+ 2815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:19.349885 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:19.351062 (+ 1177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:19.357462 (+ 6400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:19.357468 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:19.357485 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:19.357967 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:19.359304 (+ 1337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:19.359312 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:19.359319 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:19.359532 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:19.360388 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:19.360394 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:19.360645 (+ 251us) client_negotiation.cc:769] Sending connection context
0925 09:27:19.362197 (+ 1552us) client_negotiation.cc:240] Negotiation successful
0925 09:27:19.365639 (+ 3442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13551,"mutex_wait_us":528,"thread_start_us":18168,"threads_started":1}
I20260925 09:27:19.368880 17612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.316292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:33261 (local address 127.0.0.1:43058)
0925 09:27:19.332602 (+ 16310us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:19.332625 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:19.332810 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:19.333456 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:19.333462 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:19.334282 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:19.334728 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.334745 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.341912 (+ 7167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.341921 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.347514 (+ 5593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.347527 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.349265 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.349271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.349438 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.351320 (+ 1882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:19.351351 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:19.352564 (+ 1213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:19.361978 (+ 9414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:19.361985 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:19.361992 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:19.362756 (+ 764us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:19.363181 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:19.363189 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:19.363195 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:19.363328 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:19.364101 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:19.364105 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:19.364256 (+ 151us) client_negotiation.cc:769] Sending connection context
0925 09:27:19.364577 (+ 321us) client_negotiation.cc:240] Negotiation successful
0925 09:27:19.366371 (+ 1794us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16140,"mutex_wait_us":1534,"thread_start_us":28681,"threads_started":1}
Sep 25 09:27:19 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328436, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.126@KRBTEST.COM for oryx/127.10.114.124@KRBTEST.COM
I20260925 09:27:19.388095 17583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.376016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:33261 (local address 127.0.0.1:43066)
0925 09:27:19.376280 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:19.376296 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:19.376422 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:19.376796 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:19.376804 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:19.377160 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:19.377281 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.377289 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.378316 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.378323 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.379186 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.379196 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.379994 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.380000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.380163 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.380814 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:19.380828 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:19.382779 (+ 1951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:19.385788 (+ 3009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:19.385794 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:19.385797 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:19.386160 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:19.386495 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:19.386501 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:19.386504 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:19.386626 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:19.387131 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:19.387136 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:19.387301 (+ 165us) client_negotiation.cc:769] Sending connection context
0925 09:27:19.387438 (+ 137us) client_negotiation.cc:240] Negotiation successful
0925 09:27:19.387687 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260925 09:27:19.388100 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.376061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43066 (local address 127.10.114.124:33261)
0925 09:27:19.376219 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:19.376224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:19.376368 (+ 144us) server_negotiation.cc:407] Connection header received
0925 09:27:19.376569 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:19.376574 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:19.376661 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:19.376763 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:19.377418 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.378110 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.379319 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.379843 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.379965 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.382948 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:19.382953 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:19.382957 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:19.382995 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:19.385639 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.386274 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.386284 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.386288 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.386395 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.386756 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.386760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.386764 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.387018 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:19.387159 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:19.387546 (+ 387us) server_negotiation.cc:299] Negotiation successful
0925 09:27:19.387745 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 09:27:19.389549 16865 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.114.124:33261: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42"
member_type: NON_VOTER
last_known_addr {
host: "127.10.114.124"
port: 33261
}
attrs {
promote: true
}
}
dest_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04"
cas_config_opid_index: 6
I20260925 09:27:19.390161 16865 consensus_queue.cc:237] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: NON_VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: true } }
I20260925 09:27:19.392302 17479 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 1 FOLLOWER]: Refusing update from remote peer de3ad4b7da3040669e2c44a8dfe9cf04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260925 09:27:19.392561 17560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:19.392946 17422 consensus_queue.cc:1035] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 09:27:19.395896 17424 raft_consensus.cc:2932] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER fe4bf2501e9144b79c313dc56a923f42 (127.10.114.124) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: NON_VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: true } } }
I20260925 09:27:19.396332 17479 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER fe4bf2501e9144b79c313dc56a923f42 (127.10.114.124) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: NON_VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: true } } }
I20260925 09:27:19.400492 17424 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.114.124:33261
I20260925 09:27:19.400492 17507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: NON_VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: true } } }
I20260925 09:27:19.400782 17507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:27:19.400820 17424 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: NON_VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: true } } }
I20260925 09:27:19.401958 17560 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 1 FOLLOWER]: Refusing update from remote peer de3ad4b7da3040669e2c44a8dfe9cf04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260925 09:27:19.402050 17427 master.cc:537] Master@127.10.114.124:33261 shutting down...
I20260925 09:27:19.401301 17422 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: NON_VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: true } } }
I20260925 09:27:19.402843 17424 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:19.402889 17422 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:19.403699 17424 consensus_queue.cc:1035] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: NON_VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:27:19.407357 16849 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260925 09:27:19.414885 16849 consensus_peers.cc:487] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 -> Peer fe4bf2501e9144b79c313dc56a923f42 (127.10.114.124:33261): Couldn't send request to peer fe4bf2501e9144b79c313dc56a923f42. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:27:19.438187 17427 raft_consensus.cc:2229] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:27:19.438567 17427 raft_consensus.cc:2258] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:19.438743 17427 tablet_replica.cc:330] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: stopping tablet replica
I20260925 09:27:19.445396 17427 master.cc:552] Master@127.10.114.124:33261 shutdown complete.
I20260925 09:27:19.445478 17427 master_runner.cc:305] Clearing existing system tablet
I20260925 09:27:19.445851 17427 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:19.445961 17427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:19.454629 17427 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.009s sys 0.001s
I20260925 09:27:19.456312 17618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:19.456619 17427 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:19.456908 17427 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 09:27:19.457027 17427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/wal
uuid: "fe4bf2501e9144b79c313dc56a923f42"
format_stamp: "Formatted at 2026-09-25 09:27:19 on dist-test-slave-9pj8"
server_key: "ebe5bfe30a9633ecc9c632940b83e9d9"
server_key_iv: "30511973841ad96d1fad98abf9e6cd39"
server_key_version: "encryptionkey@0"
I20260925 09:27:19.463258 17427 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:19.465301 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 16926
I20260925 09:27:19.471200 17427 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 09:27:19.471352 17427 log.cc:1196] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260925 09:27:19.471833 17427 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: Deleting consensus metadata
I20260925 09:27:19.472098 17427 master_runner.cc:315] Copying system tablet from 127.10.114.126:43419
I20260925 09:27:19.477571 17427 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: tablet copy: Beginning tablet copy session from remote peer at address 127.10.114.126:43419
I20260925 09:27:19.487398 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:38265
--local_ip_for_outbound_sockets=127.10.114.65
--tserver_master_addrs=127.10.114.126:43419
--webserver_port=35697
--webserver_interface=127.10.114.65
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.65.keytab
--principal=oryx/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:39685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.114.126:43419,127.10.114.125:33507
--tserver_master_addrs=127.10.114.126:43419,127.10.114.125:33507,127.10.114.124:33261 with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:27:19.498721 17624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.478307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.0.0.1:52606)
0925 09:27:19.486517 (+ 8210us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:19.486543 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:19.486713 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:19.487059 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:19.487063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:19.487505 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:19.487718 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.487727 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.488844 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.488850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.489762 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.489772 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.490428 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.490433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.490620 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.491229 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:19.491246 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:19.492160 (+ 914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:19.495950 (+ 3790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:19.495955 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:19.495959 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:19.496516 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:19.496919 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:19.496924 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:19.496928 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:19.497097 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:19.497720 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:19.497725 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:19.497897 (+ 172us) client_negotiation.cc:769] Sending connection context
0925 09:27:19.498030 (+ 133us) client_negotiation.cc:240] Negotiation successful
0925 09:27:19.498286 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8068,"thread_start_us":7944,"threads_started":1}
I20260925 09:27:19.499177 17056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.478389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52606 (local address 127.10.114.126:43419)
0925 09:27:19.478632 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:19.478640 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:19.486642 (+ 8002us) server_negotiation.cc:407] Connection header received
0925 09:27:19.486857 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:19.486862 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:19.486933 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:19.487037 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:19.487894 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.488714 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.489895 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.490272 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.490391 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.492361 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:19.492369 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:19.492378 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:19.492407 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:19.495595 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.496645 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.496652 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.496659 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.496810 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.497220 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.497227 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.497233 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.497607 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:19.497765 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:19.498163 (+ 398us) server_negotiation.cc:299] Negotiation successful
0925 09:27:19.498409 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 09:27:19.499856 16885 tablet_copy_service.cc:140] P de3ad4b7da3040669e2c44a8dfe9cf04: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer fe4bf2501e9144b79c313dc56a923f42 ({username='oryx', principal='oryx/127.10.114.124@KRBTEST.COM'} at 127.0.0.1:52606)
I20260925 09:27:19.500063 16885 tablet_copy_service.cc:161] P de3ad4b7da3040669e2c44a8dfe9cf04: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer fe4bf2501e9144b79c313dc56a923f42 at {username='oryx', principal='oryx/127.10.114.124@KRBTEST.COM'} at 127.0.0.1:52606: session id = fe4bf2501e9144b79c313dc56a923f42-00000000000000000000000000000000
I20260925 09:27:19.503651 16885 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 09:27:19.505636 17427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 09:27:19.512965 17427 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: tablet copy: Starting download of 0 data blocks...
I20260925 09:27:19.513314 17427 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: tablet copy: Starting download of 1 WAL segments...
I20260925 09:27:19.515426 17427 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 09:27:19.519549 16885 tablet_copy_service.cc:342] P de3ad4b7da3040669e2c44a8dfe9cf04: Request end of tablet copy session fe4bf2501e9144b79c313dc56a923f42-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.114.124@KRBTEST.COM'} at 127.0.0.1:52606
I20260925 09:27:19.519670 16885 tablet_copy_service.cc:433] P de3ad4b7da3040669e2c44a8dfe9cf04: ending tablet copy session fe4bf2501e9144b79c313dc56a923f42-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer fe4bf2501e9144b79c313dc56a923f42
I20260925 09:27:19.534502 17427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:19.537173 17626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:19.541141 17627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:19.548458 17629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:19.548826 17427 server_base.cc:1034] running on GCE node
I20260925 09:27:19.549149 17427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:19.549741 17427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:19.550944 17427 hybrid_clock.cc:648] HybridClock initialized: now 1790328439550883 us; error 45 us; skew 500 ppm
I20260925 09:27:19.551157 17427 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:19 dist-test-slave-9pj8 krb5kdc[16820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328439, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:19.553143 17427 init.cc:377] Logged in from keytab as oryx/127.10.114.124@KRBTEST.COM (short username oryx)
I20260925 09:27:19.560956 17427 webserver.cc:460] Webserver started at http://127.10.114.124:36399/ using document root <none> and password file <none>
I20260925 09:27:19.561305 17427 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:19.561406 17427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:19.564393 17427 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260925 09:27:19.566154 17636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:19.566524 17427 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 09:27:19.566668 17427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/wal
uuid: "fe4bf2501e9144b79c313dc56a923f42"
format_stamp: "Formatted at 2026-09-25 09:27:19 on dist-test-slave-9pj8"
server_key: "ebe5bfe30a9633ecc9c632940b83e9d9"
server_key_iv: "30511973841ad96d1fad98abf9e6cd39"
server_key_version: "encryptionkey@0"
I20260925 09:27:19.566888 17427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:19.588068 17427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:19.591703 17427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:19.646234 17427 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.124:33261
I20260925 09:27:19.646299 17687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.124:33261 every 8 connection(s)
I20260925 09:27:19.647451 17427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/master-2/data/info.pb
I20260925 09:27:19.653143 17688 sys_catalog.cc:263] Verifying existing consensus state
I20260925 09:27:19.657074 17688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: Bootstrap starting.
I20260925 09:27:19.676770 17688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 09:27:19.677277 17688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: Bootstrap complete.
I20260925 09:27:19.678311 17688 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: NON_VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: true } }
I20260925 09:27:19.678594 17688 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: fe4bf2501e9144b79c313dc56a923f42, State: Initialized, Role: LEARNER
I20260925 09:27:19.678834 17688 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: NON_VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: true } }
I20260925 09:27:19.679913 17688 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:27:19.679644 17691 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: NON_VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: true } } }
I20260925 09:27:19.680186 17691 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [sys.catalog]: This master's current role is: LEARNER
I20260925 09:27:19.687767 17702 catalog_manager.cc:1259] Loaded cluster ID: 5344fdedc2704f5782bd1313986aaf4e
I20260925 09:27:19.687865 17702 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: loading cluster ID for follower catalog manager: success
I20260925 09:27:19.691471 17702 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: acquiring CA information for follower catalog manager: success
I20260925 09:27:19.693816 17702 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 09:27:19.846668 17625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:19.846962 17625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:19.847118 17625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:19.853747 17625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:19.853868 17625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:19.854049 17625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:27:19.862774 17625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:39685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.65.keytab
--principal=oryx/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:38265
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=35697
--tserver_master_addrs=127.10.114.126:43419,127.10.114.125:33507,127.10.114.124:33261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:19.864050 17625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:19.865659 17625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:19.878057 17708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:19.878093 17710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:19.878093 17707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:19.878873 17625 server_base.cc:1034] running on GCE node
I20260925 09:27:19.879498 17625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:19.880497 17625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:19.881736 17625 hybrid_clock.cc:648] HybridClock initialized: now 1790328439881685 us; error 34 us; skew 500 ppm
I20260925 09:27:19.882067 17625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:19 dist-test-slave-9pj8 krb5kdc[16820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328439, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:19.885576 17625 init.cc:377] Logged in from keytab as oryx/127.10.114.65@KRBTEST.COM (short username oryx)
I20260925 09:27:19.887143 17625 webserver.cc:460] Webserver started at http://127.10.114.65:35697/ using document root <none> and password file <none>
I20260925 09:27:19.887898 17625 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:19.888047 17625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:19.892974 17625 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:27:19.895491 17717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:19.896790 17625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:27:19.896945 17625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "5cd99001c7384aaab17cb23748b9611a"
format_stamp: "Formatted at 2026-09-25 09:27:17 on dist-test-slave-9pj8"
server_key: "cfae12940f3cf6d549dc107b0e63778b"
server_key_iv: "25c9aab81c88662e329acf229f640451"
server_key_version: "encryptionkey@0"
I20260925 09:27:19.897888 17625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:19.926644 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.919152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:33261 (local address 127.0.0.1:43076)
0925 09:27:19.920541 (+ 1389us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:19.920556 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:19.920664 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:19.921003 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:19.921006 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:19.921024 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:19.921167 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.921173 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.922845 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.922848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.924049 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.924060 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.926067 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.926072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.926201 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.926222 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 09:27:19.926337 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 09:27:19.926432 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1280,"thread_start_us":1211,"threads_started":1}
I20260925 09:27:19.926833 17725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.919241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43076 (local address 127.10.114.124:33261)
0925 09:27:19.919821 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:19.919832 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:19.920638 (+ 806us) server_negotiation.cc:407] Connection header received
0925 09:27:19.920779 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:19.920784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:19.920869 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:19.920983 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:19.921276 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.922649 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.924197 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.925827 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.925987 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.926156 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:19.926384 (+ 228us) server_negotiation.cc:299] Negotiation successful
0925 09:27:19.926511 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":839,"threads_started":1}
I20260925 09:27:19.927692 17663 raft_consensus.cc:1203] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260925 09:27:19.927922 17625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:19.928339 17691 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de3ad4b7da3040669e2c44a8dfe9cf04. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: NON_VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: true } } }
I20260925 09:27:19.928637 17691 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [sys.catalog]: This master's current role is: LEARNER
I20260925 09:27:19.932349 17625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:19.932618 17625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:19.933494 17625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:19.934751 17625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:19.934857 17625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:19.934971 17625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:19.935077 17625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:19.966877 17625 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:38265
I20260925 09:27:19.966954 17833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:38265 every 8 connection(s)
I20260925 09:27:19.968453 17625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:27:19.970240 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 17625
I20260925 09:27:19.970641 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 17060
Sep 25 09:27:19 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328439, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM
I20260925 09:27:20.000548 17056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.973617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:36641 (local address 127.10.114.126:43419)
0925 09:27:19.973784 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:19.973791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:19.974152 (+ 361us) server_negotiation.cc:407] Connection header received
0925 09:27:19.977619 (+ 3467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:19.977628 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:19.977733 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:19.977816 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:19.980598 (+ 2782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.982021 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.985650 (+ 3629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.986555 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.986690 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.992471 (+ 5781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:19.992481 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:19.992488 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:19.992538 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:19.996809 (+ 4271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.997890 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.997897 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.997904 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.998052 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:19.998524 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:19.998528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:19.998532 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:19.998840 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:19.999009 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:19.999961 (+ 952us) server_negotiation.cc:299] Negotiation successful
0925 09:27:20.000199 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Sep 25 09:27:19 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.114.66@KRBTEST.COM
I20260925 09:27:20.002430 17840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.972342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.65:36641)
0925 09:27:19.974001 (+ 1659us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:19.974043 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:19.977225 (+ 3182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:19.978463 (+ 1238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:19.978478 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:19.979387 (+ 909us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:19.980366 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.980396 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.982245 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.982252 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.985429 (+ 3177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.985464 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.987076 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.987084 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:19.987507 (+ 423us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:19.988389 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:19.988428 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:19.992138 (+ 3710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:19.997000 (+ 4862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:19.997011 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:19.997032 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:19.997754 (+ 722us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:19.998192 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:19.998196 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:19.998200 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:19.998410 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:19.999097 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:19.999105 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:19.999531 (+ 426us) client_negotiation.cc:769] Sending connection context
0925 09:27:20.000745 (+ 1214us) client_negotiation.cc:240] Negotiation successful
0925 09:27:20.001200 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1309,"threads_started":1}
I20260925 09:27:20.005236 17835 heartbeater.cc:347] Connected to a master server at 127.10.114.126:43419
I20260925 09:27:20.005528 17835 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:20.006839 17835 heartbeater.cc:511] Master 127.10.114.126:43419 requested a full tablet report, sending...
I20260925 09:27:20.009570 16865 ts_manager.cc:194] Re-registered known tserver with Master: 5cd99001c7384aaab17cb23748b9611a (127.10.114.65:38265)
Sep 25 09:27:20 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328439, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.125@KRBTEST.COM
I20260925 09:27:20.010664 16865 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:36641
Sep 25 09:27:20 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328439, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.124@KRBTEST.COM
I20260925 09:27:20.011297 17841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.973282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56354 (local address 127.10.114.66:36841)
0925 09:27:19.973823 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:19.973834 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:19.973863 (+ 29us) server_negotiation.cc:407] Connection header received
0925 09:27:19.973936 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:19.973944 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:19.974194 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:19.976727 (+ 2533us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:19.991314 (+ 14587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.993182 (+ 1868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.995428 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:19.996345 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:19.997957 (+ 1612us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:20.002942 (+ 4985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:20.002954 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:20.002969 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:20.003006 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:20.007364 (+ 4358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:20.008184 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:20.008192 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:20.008195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:20.008320 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:20.008769 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:20.008774 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:20.008777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:20.009179 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:20.010480 (+ 1301us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:20.010640 (+ 160us) server_negotiation.cc:299] Negotiation successful
0925 09:27:20.010889 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":937,"threads_started":1}
I20260925 09:27:20.012727 17183 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:20.018435 17725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.971738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:48715 (local address 127.10.114.124:33261)
0925 09:27:19.971944 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:19.971950 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:19.996130 (+ 24180us) server_negotiation.cc:407] Connection header received
0925 09:27:19.996360 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:19.996365 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:19.996445 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:19.996505 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:20.000136 (+ 3631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.001239 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.007223 (+ 5984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.007898 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.008027 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:20.012103 (+ 4076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:20.012109 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:20.012113 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:20.012158 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:20.015249 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:20.016384 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:20.016393 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:20.016398 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:20.016555 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:20.017017 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:20.017026 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:20.017030 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:20.017367 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:20.017521 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:20.017876 (+ 355us) server_negotiation.cc:299] Negotiation successful
0925 09:27:20.018082 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260925 09:27:20.018544 17839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.974025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:33261 (local address 127.10.114.65:48715)
0925 09:27:19.996040 (+ 22015us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:19.996056 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:19.996220 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:19.996683 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:19.996687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:19.997290 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:19.997585 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.997595 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.001567 (+ 3972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.001573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:20.007061 (+ 5488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:20.007075 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.008291 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.008297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:20.008465 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:20.009412 (+ 947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:20.009432 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:20.011931 (+ 2499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:20.015376 (+ 3445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:20.015383 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:20.015389 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:20.016234 (+ 845us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:20.016717 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:20.016724 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:20.016730 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:20.016874 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:20.017491 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:20.017496 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:20.017639 (+ 143us) client_negotiation.cc:769] Sending connection context
0925 09:27:20.017776 (+ 137us) client_negotiation.cc:240] Negotiation successful
0925 09:27:20.018048 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21904,"thread_start_us":21484,"threads_started":1}
I20260925 09:27:20.019930 17834 heartbeater.cc:347] Connected to a master server at 127.10.114.124:33261
I20260925 09:27:20.020063 17834 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:20.020363 17842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.973719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:54435 (local address 127.10.114.125:33507)
0925 09:27:19.997979 (+ 24260us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:19.997984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:19.998009 (+ 25us) server_negotiation.cc:407] Connection header received
0925 09:27:19.998084 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:19.998090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:19.998620 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:19.998731 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:20.002661 (+ 3930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.003500 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.005205 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.005942 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.007926 (+ 1984us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:20.010712 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:20.010718 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:20.010723 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:20.010830 (+ 107us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:20.017096 (+ 6266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:20.017831 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:20.017837 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:20.017841 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:20.017972 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:20.018406 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:20.018411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:20.018415 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:20.019217 (+ 802us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:20.019382 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:20.019769 (+ 387us) server_negotiation.cc:299] Negotiation successful
0925 09:27:20.019995 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24148,"thread_start_us":24066,"threads_started":1}
I20260925 09:27:20.020387 17843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:19.973268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33507 (local address 127.10.114.65:54435)
0925 09:27:19.995312 (+ 22044us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:19.995333 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:19.995435 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:19.998952 (+ 3517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:19.998958 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:19.999552 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:19.999934 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:19.999949 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.003627 (+ 3678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.003635 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:20.005032 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:20.005043 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.006149 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.006155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:20.006360 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:20.007057 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:20.007076 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:20.010494 (+ 3418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:20.017222 (+ 6728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:20.017227 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:20.017231 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:20.017697 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:20.018094 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:20.018099 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:20.018103 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:20.018283 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:20.019343 (+ 1060us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:20.019346 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:20.019498 (+ 152us) client_negotiation.cc:769] Sending connection context
0925 09:27:20.019633 (+ 135us) client_negotiation.cc:240] Negotiation successful
0925 09:27:20.019964 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21910,"mutex_wait_us":1,"thread_start_us":568,"threads_started":1}
I20260925 09:27:20.020521 17834 heartbeater.cc:511] Master 127.10.114.124:33261 requested a full tablet report, sending...
I20260925 09:27:20.021395 17836 heartbeater.cc:347] Connected to a master server at 127.10.114.125:33507
I20260925 09:27:20.021535 17836 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:20.021819 17836 heartbeater.cc:511] Master 127.10.114.125:33507 requested a full tablet report, sending...
I20260925 09:27:20.022336 17653 ts_manager.cc:194] Registered new tserver with Master: 5cd99001c7384aaab17cb23748b9611a (127.10.114.65:38265)
I20260925 09:27:20.023293 17469 ts_manager.cc:194] Registered new tserver with Master: 5cd99001c7384aaab17cb23748b9611a (127.10.114.65:38265)
I20260925 09:27:20.097354 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 17060
I20260925 09:27:20.113582 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:36841
--local_ip_for_outbound_sockets=127.10.114.66
--tserver_master_addrs=127.10.114.126:43419
--webserver_port=34287
--webserver_interface=127.10.114.66
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.66.keytab
--principal=oryx/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:39685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.114.126:43419,127.10.114.125:33507
--tserver_master_addrs=127.10.114.126:43419,127.10.114.125:33507,127.10.114.124:33261 with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:27:20.361281 17723 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: NON_VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: true } } }
I20260925 09:27:20.361773 17723 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:20.362805 17846 raft_consensus.cc:1050] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04: attempting to promote NON_VOTER fe4bf2501e9144b79c313dc56a923f42 to VOTER
I20260925 09:27:20.363533 17846 consensus_queue.cc:237] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: false } }
I20260925 09:27:20.365588 17663 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 1 LEARNER]: Refusing update from remote peer de3ad4b7da3040669e2c44a8dfe9cf04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260925 09:27:20.365711 17479 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 1 FOLLOWER]: Refusing update from remote peer de3ad4b7da3040669e2c44a8dfe9cf04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260925 09:27:20.366252 17846 consensus_queue.cc:1035] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260925 09:27:20.366636 17722 consensus_queue.cc:1035] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260925 09:27:20.369143 17846 raft_consensus.cc:2932] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, fe4bf2501e9144b79c313dc56a923f42 (127.10.114.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: false } } }
I20260925 09:27:20.369722 17479 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, fe4bf2501e9144b79c313dc56a923f42 (127.10.114.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: false } } }
I20260925 09:27:20.374388 17723 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: false } } }
I20260925 09:27:20.374686 17723 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:20.375097 17723 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: false } } }
I20260925 09:27:20.375424 17723 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:20.383270 17722 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: false } } }
I20260925 09:27:20.383558 17722 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:20.383960 17663 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, fe4bf2501e9144b79c313dc56a923f42 (127.10.114.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: false } } }
I20260925 09:27:20.384527 17849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: false } } }
I20260925 09:27:20.384902 17849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:27:20.386869 17691 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: false } } }
I20260925 09:27:20.387111 17691 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 09:27:20.438170 17845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:20.438469 17845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:20.438634 17845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:20.445910 17845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:20.446044 17845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:20.446223 17845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:27:20.455387 17845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:39685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.66.keytab
--principal=oryx/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:36841
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=34287
--tserver_master_addrs=127.10.114.126:43419,127.10.114.125:33507,127.10.114.124:33261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:20.456647 17845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:20.457944 17845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:20.470399 17854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:20.470359 17855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:20.471000 17857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:20.471503 17845 server_base.cc:1034] running on GCE node
I20260925 09:27:20.472117 17845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:20.472919 17845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:20.474146 17845 hybrid_clock.cc:648] HybridClock initialized: now 1790328440474110 us; error 46 us; skew 500 ppm
I20260925 09:27:20.474486 17845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:20 dist-test-slave-9pj8 krb5kdc[16820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328440, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:20.477591 17845 init.cc:377] Logged in from keytab as oryx/127.10.114.66@KRBTEST.COM (short username oryx)
I20260925 09:27:20.479053 17845 webserver.cc:460] Webserver started at http://127.10.114.66:34287/ using document root <none> and password file <none>
I20260925 09:27:20.479652 17845 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:20.479831 17845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:20.485076 17845 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260925 09:27:20.487638 17864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:20.488941 17845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:27:20.489099 17845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "32afe2fd0d664cb588b6b4a2740e1106"
format_stamp: "Formatted at 2026-09-25 09:27:17 on dist-test-slave-9pj8"
server_key: "715b38b0a376311a799e27ab00ec28ae"
server_key_iv: "cd848415eb9a74d668a9039e953d5938"
server_key_version: "encryptionkey@0"
I20260925 09:27:20.489976 17845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:20.506232 17845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:20.510862 17845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:20.511178 17845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:20.512190 17845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:20.513525 17845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:20.513626 17845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:20.513725 17845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:20.513804 17845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:20.547446 17845 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:36841
I20260925 09:27:20.547477 17976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:36841 every 8 connection(s)
I20260925 09:27:20.548879 17845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:27:20.551576 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 17845
I20260925 09:27:20.551862 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 17193
Sep 25 09:27:20 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328440, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.124@KRBTEST.COM
Sep 25 09:27:20 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328440, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM
Sep 25 09:27:20 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.114.67@KRBTEST.COM
I20260925 09:27:20.594949 17986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:20.554437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:47959 (local address 127.10.114.124:33261)
0925 09:27:20.568556 (+ 14119us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:20.568562 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:20.568582 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:27:20.570524 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:20.570533 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:20.570639 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:20.570736 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:20.575450 (+ 4714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.576947 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.580506 (+ 3559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.581135 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.581327 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:20.586220 (+ 4893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:20.586227 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:20.586231 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:20.586272 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:20.589963 (+ 3691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:20.591846 (+ 1883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:20.591852 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:20.591855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:20.592002 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:20.592658 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:20.592664 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:20.592667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:20.593041 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:20.593165 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:20.594267 (+ 1102us) server_negotiation.cc:299] Negotiation successful
0925 09:27:20.594488 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13998,"thread_start_us":13868,"threads_started":1}
Sep 25 09:27:20 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328440, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.125@KRBTEST.COM
I20260925 09:27:20.601797 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:20.553454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51134 (local address 127.10.114.67:46759)
0925 09:27:20.569126 (+ 15672us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:20.569135 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:20.569167 (+ 32us) server_negotiation.cc:407] Connection header received
0925 09:27:20.569239 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:20.569249 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:20.572222 (+ 2973us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:20.572416 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:20.575947 (+ 3531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.577910 (+ 1963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.582864 (+ 4954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.584190 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.584333 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:20.593058 (+ 8725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:20.593075 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:20.593095 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:20.593134 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:20.597809 (+ 4675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:20.598602 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:20.598612 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:20.598618 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:20.599037 (+ 419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:20.599467 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:20.599472 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:20.599476 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:20.599931 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:20.600184 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:20.601056 (+ 872us) server_negotiation.cc:299] Negotiation successful
0925 09:27:20.601397 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15507,"thread_start_us":15300,"threads_started":1}
I20260925 09:27:20.604123 17316 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:20.607501 17988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:20.569948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:54039 (local address 127.10.114.125:33507)
0925 09:27:20.585158 (+ 15210us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:20.585166 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:20.585185 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:27:20.585253 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:20.585259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:20.585357 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:20.585464 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:20.587214 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.590788 (+ 3574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.592976 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.593697 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.594514 (+ 817us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:20.598096 (+ 3582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:20.598102 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:20.598106 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:20.598144 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:20.602305 (+ 4161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:20.603139 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:20.603144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:20.603147 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:20.603270 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:20.603718 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:20.603724 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:20.603730 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:20.604118 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:20.604263 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:20.605765 (+ 1502us) server_negotiation.cc:299] Negotiation successful
0925 09:27:20.606248 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15066,"thread_start_us":15606,"threads_started":1}
I20260925 09:27:20.608282 17987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:20.569041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33507 (local address 127.10.114.66:54039)
0925 09:27:20.579116 (+ 10075us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:20.579134 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:20.579282 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:20.585833 (+ 6551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:20.585838 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:20.586411 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:20.587025 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:20.587041 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.590969 (+ 3928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.590975 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:20.592793 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:20.592811 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.593971 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.593979 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:20.594232 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:20.594768 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:20.594791 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:20.597853 (+ 3062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:20.602473 (+ 4620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:20.602478 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:20.602483 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:20.603009 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:20.603411 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:20.603416 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:20.603419 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:20.603580 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:20.604356 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:20.604377 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:20.604542 (+ 165us) client_negotiation.cc:769] Sending connection context
0925 09:27:20.604711 (+ 169us) client_negotiation.cc:240] Negotiation successful
0925 09:27:20.605245 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9952,"mutex_wait_us":219,"thread_start_us":10305,"threads_started":1}
I20260925 09:27:20.608400 17982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:20.553222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:33261 (local address 127.10.114.66:47959)
0925 09:27:20.565038 (+ 11816us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:20.565098 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:20.569794 (+ 4696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:20.571438 (+ 1644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:20.571454 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:20.572825 (+ 1371us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:20.573823 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:20.573854 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.577210 (+ 3356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.577218 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:20.580340 (+ 3122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:20.580350 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.581720 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.581729 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:20.582084 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:20.582968 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:20.583001 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:20.585939 (+ 2938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:20.590115 (+ 4176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:20.590124 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:20.590142 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:20.591677 (+ 1535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:20.592169 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:20.592176 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:20.592183 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:20.592498 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:20.593358 (+ 860us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:20.593365 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:20.593793 (+ 428us) client_negotiation.cc:769] Sending connection context
0925 09:27:20.606856 (+ 13063us) client_negotiation.cc:240] Negotiation successful
0925 09:27:20.607294 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11432,"thread_start_us":11165,"threads_started":1}
I20260925 09:27:20.610033 17977 heartbeater.cc:347] Connected to a master server at 127.10.114.124:33261
I20260925 09:27:20.610369 17985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:20.553605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:46039 (local address 127.10.114.126:43419)
0925 09:27:20.569257 (+ 15652us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:20.569265 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:20.569287 (+ 22us) server_negotiation.cc:407] Connection header received
0925 09:27:20.579691 (+ 10404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:20.579699 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:20.580167 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:20.580318 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:20.581319 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.582931 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.585591 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.586308 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.586442 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:20.591434 (+ 4992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:20.591442 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:20.591449 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:20.591492 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:20.604468 (+ 12976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:20.607456 (+ 2988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:20.607463 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:20.607468 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:20.607627 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:20.608234 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:20.608240 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:20.608243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:20.608583 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:20.608717 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:20.609830 (+ 1113us) server_negotiation.cc:299] Negotiation successful
0925 09:27:20.610039 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15520,"thread_start_us":17800,"threads_started":1}
I20260925 09:27:20.610416 17977 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:20.610751 17979 heartbeater.cc:347] Connected to a master server at 127.10.114.125:33507
I20260925 09:27:20.610898 17979 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:20.611537 17977 heartbeater.cc:511] Master 127.10.114.124:33261 requested a full tablet report, sending...
I20260925 09:27:20.611563 17979 heartbeater.cc:511] Master 127.10.114.125:33507 requested a full tablet report, sending...
I20260925 09:27:20.612838 17984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:20.553222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.66:46039)
0925 09:27:20.566735 (+ 13513us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:20.566752 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:20.579519 (+ 12767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:20.580332 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:20.580336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:20.580899 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:20.581159 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:20.581172 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.583041 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.583047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:20.585457 (+ 2410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:20.585472 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:20.586506 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:20.586512 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:20.586734 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:20.587414 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:20.587437 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:20.591260 (+ 3823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:20.606008 (+ 14748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:20.606017 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:20.606023 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:20.606714 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:20.607884 (+ 1170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:20.607892 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:20.607898 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:20.608081 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:20.609206 (+ 1125us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:20.609210 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:20.609379 (+ 169us) client_negotiation.cc:769] Sending connection context
0925 09:27:20.609558 (+ 179us) client_negotiation.cc:240] Negotiation successful
0925 09:27:20.612321 (+ 2763us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13343,"mutex_wait_us":62,"spinlock_wait_cycles":42880,"thread_start_us":13449,"threads_started":1}
I20260925 09:27:20.613406 17469 ts_manager.cc:194] Registered new tserver with Master: 32afe2fd0d664cb588b6b4a2740e1106 (127.10.114.66:36841)
I20260925 09:27:20.613519 17653 ts_manager.cc:194] Registered new tserver with Master: 32afe2fd0d664cb588b6b4a2740e1106 (127.10.114.66:36841)
I20260925 09:27:20.614307 17978 heartbeater.cc:347] Connected to a master server at 127.10.114.126:43419
I20260925 09:27:20.614859 17978 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:20.615345 17978 heartbeater.cc:511] Master 127.10.114.126:43419 requested a full tablet report, sending...
I20260925 09:27:20.616395 16865 ts_manager.cc:194] Re-registered known tserver with Master: 32afe2fd0d664cb588b6b4a2740e1106 (127.10.114.66:36841)
I20260925 09:27:20.617761 16865 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:46039
I20260925 09:27:20.700350 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 17193
I20260925 09:27:20.716568 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:46759
--local_ip_for_outbound_sockets=127.10.114.67
--tserver_master_addrs=127.10.114.126:43419
--webserver_port=38423
--webserver_interface=127.10.114.67
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.67.keytab
--principal=oryx/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:39685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.114.126:43419,127.10.114.125:33507
--tserver_master_addrs=127.10.114.126:43419,127.10.114.125:33507,127.10.114.124:33261 with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:27:21.014810 17835 heartbeater.cc:503] Master 127.10.114.126:43419 was elected leader, sending a full tablet report...
W20260925 09:27:21.036300 17990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:21.036563 17990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:21.036737 17990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:21.043898 17990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:21.043991 17990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:21.044165 17990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:27:21.053396 17990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:39685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.67.keytab
--principal=oryx/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:46759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=38423
--tserver_master_addrs=127.10.114.126:43419,127.10.114.125:33507,127.10.114.124:33261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:21.054663 17990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:21.055959 17990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:21.068890 17996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:21.069089 17995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:21.069285 17998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:21.070283 17990 server_base.cc:1034] running on GCE node
I20260925 09:27:21.070834 17990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:21.071688 17990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:21.072917 17990 hybrid_clock.cc:648] HybridClock initialized: now 1790328441072867 us; error 41 us; skew 500 ppm
I20260925 09:27:21.073232 17990 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:21 dist-test-slave-9pj8 krb5kdc[16820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328441, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:21.076488 17990 init.cc:377] Logged in from keytab as oryx/127.10.114.67@KRBTEST.COM (short username oryx)
I20260925 09:27:21.077913 17990 webserver.cc:460] Webserver started at http://127.10.114.67:38423/ using document root <none> and password file <none>
I20260925 09:27:21.078570 17990 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:21.078722 17990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:21.084131 17990 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:27:21.086726 18005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:21.088212 17990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:27:21.088380 17990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "2dbbfd760b8240e98b652fd145f7df82"
format_stamp: "Formatted at 2026-09-25 09:27:18 on dist-test-slave-9pj8"
server_key: "891fb3dc522934033684b7be895a296d"
server_key_iv: "09c0c5ed7738fa9e5caee2479fb7faf0"
server_key_version: "encryptionkey@0"
I20260925 09:27:21.089359 17990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:21.104591 17990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:21.108430 17990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:21.108700 17990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:21.109670 17990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:21.110917 17990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:21.111037 17990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:21.111150 17990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:21.111274 17990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:21.143162 17990 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:46759
I20260925 09:27:21.143254 18117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:46759 every 8 connection(s)
I20260925 09:27:21.144639 17990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:27:21.155010 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 17990
Sep 25 09:27:21 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328441, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.125@KRBTEST.COM
Sep 25 09:27:21 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328441, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM
Sep 25 09:27:21 dist-test-slave-9pj8 krb5kdc[16820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328441, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.124@KRBTEST.COM
I20260925 09:27:21.217550 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.181010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43084 (local address 127.10.114.124:33261)
0925 09:27:21.196205 (+ 15195us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:21.196211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:21.197087 (+ 876us) server_negotiation.cc:407] Connection header received
0925 09:27:21.197158 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:21.197163 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:21.197243 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:21.197348 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:21.200536 (+ 3188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.202418 (+ 1882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.205084 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.205747 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.205888 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.209875 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:21.209884 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:21.209891 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:21.209937 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:21.213625 (+ 3688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:21.214560 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:21.214566 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:21.214570 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:21.214780 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:21.215383 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:21.215389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:21.215392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:21.215820 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:21.216090 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:21.216806 (+ 716us) server_negotiation.cc:299] Negotiation successful
0925 09:27:21.217043 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15059,"thread_start_us":15962,"threads_started":1}
I20260925 09:27:21.223204 18123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.147560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:42003 (local address 127.10.114.126:43419)
0925 09:27:21.202824 (+ 55264us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:21.202831 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:21.202850 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:27:21.202909 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:21.202916 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:21.202992 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:21.203117 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:21.205135 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.206288 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.208359 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.209092 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.209211 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.214013 (+ 4802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:21.214019 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:21.214024 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:21.214060 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:21.217977 (+ 3917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:21.219180 (+ 1203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:21.219187 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:21.219193 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:21.219356 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:21.219857 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:21.219863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:21.219866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:21.220307 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:21.220468 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:21.222288 (+ 1820us) server_negotiation.cc:299] Negotiation successful
0925 09:27:21.222508 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55145,"thread_start_us":55037,"threads_started":1}
I20260925 09:27:21.223552 18125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.148164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.67:42003)
0925 09:27:21.162441 (+ 14277us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:21.162462 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:21.165423 (+ 2961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:21.203171 (+ 37748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:21.203177 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:21.203848 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:21.204188 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:21.204201 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.206405 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.206411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:21.208182 (+ 1771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:21.208200 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.209272 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.209278 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:21.209501 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.210457 (+ 956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:21.210487 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:21.213234 (+ 2747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:21.218435 (+ 5201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:21.218445 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:21.218464 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:21.219026 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:21.219468 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:21.219476 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:21.219480 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:21.219696 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:21.220626 (+ 930us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:21.220634 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:21.221062 (+ 428us) client_negotiation.cc:769] Sending connection context
0925 09:27:21.221424 (+ 362us) client_negotiation.cc:240] Negotiation successful
0925 09:27:21.222007 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14068,"thread_start_us":49280,"threads_started":1}
I20260925 09:27:21.223888 18126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.148165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33507 (local address 127.10.114.67:43389)
0925 09:27:21.160013 (+ 11848us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:21.160070 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:21.165028 (+ 4958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:21.183021 (+ 17993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:21.183040 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:21.184147 (+ 1107us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:21.185245 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:21.185275 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.190717 (+ 5442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.190725 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:21.193699 (+ 2974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:21.193717 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.195977 (+ 2260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.195983 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:21.196341 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.197206 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:21.197244 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:21.200733 (+ 3489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:21.220985 (+ 20252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:21.220989 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:21.221002 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:21.221483 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:21.221942 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:21.221948 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:21.221951 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:21.222111 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:21.222744 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:21.222750 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:21.222925 (+ 175us) client_negotiation.cc:769] Sending connection context
0925 09:27:21.223048 (+ 123us) client_negotiation.cc:240] Negotiation successful
0925 09:27:21.223345 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10905,"spinlock_wait_cycles":34432,"thread_start_us":14251,"threads_started":1}
I20260925 09:27:21.225587 18133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.147400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:43389 (local address 127.10.114.125:33507)
0925 09:27:21.180713 (+ 33313us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:21.180721 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:21.180744 (+ 23us) server_negotiation.cc:407] Connection header received
0925 09:27:21.181482 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:21.181491 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:21.181662 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:21.185516 (+ 3854us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:21.185718 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.187199 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.193946 (+ 6747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.194855 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.201032 (+ 6177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.202194 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:21.202205 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:21.202211 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:21.202258 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:21.217717 (+ 15459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:21.221626 (+ 3909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:21.221631 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:21.221633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:21.221814 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:21.222259 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:21.222265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:21.222269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:21.222610 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:21.224014 (+ 1404us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:21.224177 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 09:27:21.225176 (+ 999us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33184,"thread_start_us":63105,"threads_started":1}
I20260925 09:27:21.225890 18119 heartbeater.cc:347] Connected to a master server at 127.10.114.126:43419
I20260925 09:27:21.226244 18119 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:21.227568 18119 heartbeater.cc:511] Master 127.10.114.126:43419 requested a full tablet report, sending...
I20260925 09:27:21.228044 18127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.148991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:37889 (local address 127.10.114.124:33261)
0925 09:27:21.158820 (+ 9829us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:21.158829 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:21.197991 (+ 39162us) server_negotiation.cc:407] Connection header received
0925 09:27:21.198144 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:21.198164 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:21.198268 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:21.198423 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:21.200413 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.202604 (+ 2191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.204574 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.205048 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.205189 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.214237 (+ 9048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:21.214245 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:21.214252 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:21.214293 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:21.218268 (+ 3975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:21.220314 (+ 2046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:21.220319 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:21.220322 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:21.220472 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:21.226270 (+ 5798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:21.226278 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:21.226285 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:21.226662 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:21.226794 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:21.227353 (+ 559us) server_negotiation.cc:299] Negotiation successful
0925 09:27:21.227579 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9663,"thread_start_us":9532,"threads_started":1}
I20260925 09:27:21.229053 18132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.169810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52614 (local address 127.10.114.126:43419)
0925 09:27:21.203811 (+ 34001us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:21.203817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:21.203838 (+ 21us) server_negotiation.cc:407] Connection header received
0925 09:27:21.203885 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:21.203890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:21.203951 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:21.208508 (+ 4557us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:21.208719 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.213398 (+ 4679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.215325 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.216132 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.216265 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.220306 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:21.220311 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:21.220315 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:21.220349 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:21.225368 (+ 5019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:21.226198 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:21.226206 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:21.226211 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:21.226361 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:21.226799 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:21.226806 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:21.226812 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:21.227171 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:21.227394 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:21.228435 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0925 09:27:21.228629 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33867,"thread_start_us":34224,"threads_started":1}
I20260925 09:27:21.229596 16865 ts_manager.cc:194] Re-registered known tserver with Master: 2dbbfd760b8240e98b652fd145f7df82 (127.10.114.67:46759)
I20260925 09:27:21.229641 18120 heartbeater.cc:347] Connected to a master server at 127.10.114.125:33507
I20260925 09:27:21.229759 18120 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:21.230206 18120 heartbeater.cc:511] Master 127.10.114.125:33507 requested a full tablet report, sending...
I20260925 09:27:21.230501 16865 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:42003
I20260925 09:27:21.230976 18124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.148959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:33261 (local address 127.10.114.67:37889)
0925 09:27:21.197883 (+ 48924us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:21.197901 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:21.198017 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:21.198731 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:21.198746 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:21.199580 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:21.200123 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:21.200146 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.202803 (+ 2657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.202807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:21.204428 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:21.204437 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.205435 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.205443 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:21.205687 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.206261 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:21.206326 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:21.213987 (+ 7661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:21.219217 (+ 5230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:21.219221 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:21.219225 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:21.220163 (+ 938us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:21.225961 (+ 5798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:21.225967 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:21.225971 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:21.226129 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:21.226879 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:21.226887 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:21.227046 (+ 159us) client_negotiation.cc:769] Sending connection context
0925 09:27:21.228219 (+ 1173us) client_negotiation.cc:240] Negotiation successful
0925 09:27:21.228513 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48795,"thread_start_us":9173,"threads_started":1}
I20260925 09:27:21.231443 17469 ts_manager.cc:194] Registered new tserver with Master: 2dbbfd760b8240e98b652fd145f7df82 (127.10.114.67:46759)
I20260925 09:27:21.232707 18118 heartbeater.cc:347] Connected to a master server at 127.10.114.124:33261
I20260925 09:27:21.232843 18118 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:21.233286 18118 heartbeater.cc:511] Master 127.10.114.124:33261 requested a full tablet report, sending...
I20260925 09:27:21.234370 17653 ts_manager.cc:194] Registered new tserver with Master: 2dbbfd760b8240e98b652fd145f7df82 (127.10.114.67:46759)
I20260925 09:27:21.235703 18122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.169833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34282 (local address 127.10.114.125:33507)
0925 09:27:21.209826 (+ 39993us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:21.209832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:21.209851 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:27:21.209903 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:21.209909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:21.210433 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:21.212428 (+ 1995us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:21.212625 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.213808 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.221626 (+ 7818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.222295 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.222951 (+ 656us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.225435 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:21.225442 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:21.225447 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:21.225478 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:21.229572 (+ 4094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:21.231356 (+ 1784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:21.231360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:21.231363 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:21.231535 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:21.232265 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:21.232270 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:21.232273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:21.233778 (+ 1505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:21.235205 (+ 1427us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:21.235306 (+ 101us) server_negotiation.cc:299] Negotiation successful
0925 09:27:21.235445 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39865,"thread_start_us":10669,"threads_started":1}
I20260925 09:27:21.234606 16864 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:27:21.242307 16864 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:27:21.273931 18150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.264047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:46759 (local address 127.0.0.1:51150)
0925 09:27:21.265435 (+ 1388us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:21.265453 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:21.265565 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:21.266376 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:21.266381 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:21.266411 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:21.266800 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:21.266810 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.268442 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.268448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:21.271549 (+ 3101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:21.271558 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.272835 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.272840 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:21.273203 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.273277 (+ 74us) client_negotiation.cc:769] Sending connection context
0925 09:27:21.273359 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 09:27:21.273480 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1260,"mutex_wait_us":23,"thread_start_us":1109,"threads_started":1}
I20260925 09:27:21.274358 18148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.260111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56362 (local address 127.10.114.66:36841)
0925 09:27:21.264192 (+ 4081us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:21.264201 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:21.264910 (+ 709us) server_negotiation.cc:407] Connection header received
0925 09:27:21.265152 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:21.265164 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:21.265402 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:21.265592 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:21.266597 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.268181 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.271480 (+ 3299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.272850 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.272981 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.273161 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:21.273883 (+ 722us) server_negotiation.cc:299] Negotiation successful
0925 09:27:21.274069 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3955,"thread_start_us":3871,"threads_started":1}
I20260925 09:27:21.274475 18146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.259862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40420 (local address 127.10.114.65:38265)
0925 09:27:21.260558 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:21.260568 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:21.264604 (+ 4036us) server_negotiation.cc:407] Connection header received
0925 09:27:21.264909 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:21.264918 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:21.265226 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:21.265437 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:21.266537 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.268139 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.271183 (+ 3044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.272674 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.272826 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.273045 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:21.274005 (+ 960us) server_negotiation.cc:299] Negotiation successful
0925 09:27:21.274177 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":369,"threads_started":1}
I20260925 09:27:21.274645 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.261081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51150 (local address 127.10.114.67:46759)
0925 09:27:21.261704 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:21.261710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:21.265691 (+ 3981us) server_negotiation.cc:407] Connection header received
0925 09:27:21.265802 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:21.265811 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:21.266011 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:21.266229 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:21.266994 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.268301 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.271690 (+ 3389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.272651 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.272798 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.273006 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:21.274195 (+ 1189us) server_negotiation.cc:299] Negotiation successful
0925 09:27:21.274351 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":304,"threads_started":1}
I20260925 09:27:21.274644 18145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.258934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:38265 (local address 127.0.0.1:40420)
0925 09:27:21.264477 (+ 5543us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:21.264497 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:21.264722 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:21.265707 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:21.265713 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:21.265740 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:21.266257 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:21.266277 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.268330 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.268336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:21.271019 (+ 2683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:21.271031 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.273202 (+ 2171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.273208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:21.273442 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.273518 (+ 76us) client_negotiation.cc:769] Sending connection context
0925 09:27:21.273610 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 09:27:21.273741 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5366,"mutex_wait_us":153,"spinlock_wait_cycles":896,"thread_start_us":764,"threads_started":1}
I20260925 09:27:21.275276 18147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.259653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:36841 (local address 127.0.0.1:56362)
0925 09:27:21.264734 (+ 5081us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:21.264755 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:21.264896 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:21.265823 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:21.265827 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:21.265857 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:21.266432 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:21.266446 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.268312 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.268319 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:21.271305 (+ 2986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:21.271317 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.273296 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.273302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:21.273503 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.273534 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 09:27:21.273610 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 09:27:21.273741 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4884,"mutex_wait_us":148,"thread_start_us":4708,"threads_started":1}
I20260925 09:27:21.278630 17769 tablet_service.cc:1467] Processing CreateTablet for tablet fb1cbb15930d4c54b3c61d9512a8e120 (DEFAULT_TABLE table=test-table [id=96fc52340c074489ae50f4846a4657ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:21.278870 17912 tablet_service.cc:1467] Processing CreateTablet for tablet fb1cbb15930d4c54b3c61d9512a8e120 (DEFAULT_TABLE table=test-table [id=96fc52340c074489ae50f4846a4657ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:21.278970 18053 tablet_service.cc:1467] Processing CreateTablet for tablet fb1cbb15930d4c54b3c61d9512a8e120 (DEFAULT_TABLE table=test-table [id=96fc52340c074489ae50f4846a4657ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:21.280135 17769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb1cbb15930d4c54b3c61d9512a8e120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:21.280190 18053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb1cbb15930d4c54b3c61d9512a8e120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:21.280644 17912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb1cbb15930d4c54b3c61d9512a8e120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:21.292814 18151 tablet_bootstrap.cc:492] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106: Bootstrap starting.
I20260925 09:27:21.295383 18153 tablet_bootstrap.cc:492] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82: Bootstrap starting.
I20260925 09:27:21.296229 18152 tablet_bootstrap.cc:492] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a: Bootstrap starting.
I20260925 09:27:21.296768 18151 tablet_bootstrap.cc:654] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:21.298481 18151 log.cc:824] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:21.300392 18152 tablet_bootstrap.cc:654] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:21.301719 18152 log.cc:824] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:21.302762 18153 tablet_bootstrap.cc:654] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:21.303999 18153 log.cc:824] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:21.304802 18152 tablet_bootstrap.cc:492] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a: No bootstrap required, opened a new log
I20260925 09:27:21.305080 18152 ts_tablet_manager.cc:1397] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260925 09:27:21.306313 18151 tablet_bootstrap.cc:492] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106: No bootstrap required, opened a new log
I20260925 09:27:21.306584 18151 ts_tablet_manager.cc:1397] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.000s
I20260925 09:27:21.309615 18153 tablet_bootstrap.cc:492] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82: No bootstrap required, opened a new log
I20260925 09:27:21.309885 18153 ts_tablet_manager.cc:1397] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.000s
I20260925 09:27:21.316386 18152 raft_consensus.cc:348] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 } } peers { permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 } } peers { permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 } }
I20260925 09:27:21.316817 18152 raft_consensus.cc:374] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:21.316921 18152 raft_consensus.cc:729] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd99001c7384aaab17cb23748b9611a, State: Initialized, Role: FOLLOWER
I20260925 09:27:21.317384 18153 raft_consensus.cc:348] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 } } peers { permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 } } peers { permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 } }
I20260925 09:27:21.317682 18153 raft_consensus.cc:374] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:21.317548 18152 consensus_queue.cc:260] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 } } peers { permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 } } peers { permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 } }
I20260925 09:27:21.317797 18153 raft_consensus.cc:729] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dbbfd760b8240e98b652fd145f7df82, State: Initialized, Role: FOLLOWER
I20260925 09:27:21.317698 18151 raft_consensus.cc:348] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 } } peers { permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 } } peers { permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 } }
I20260925 09:27:21.318109 18151 raft_consensus.cc:374] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:21.318293 18151 raft_consensus.cc:729] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32afe2fd0d664cb588b6b4a2740e1106, State: Initialized, Role: FOLLOWER
I20260925 09:27:21.318369 18153 consensus_queue.cc:260] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 } } peers { permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 } } peers { permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 } }
I20260925 09:27:21.319056 18151 consensus_queue.cc:260] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 } } peers { permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 } } peers { permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 } }
I20260925 09:27:21.319536 18152 ts_tablet_manager.cc:1428] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260925 09:27:21.326962 18153 ts_tablet_manager.cc:1428] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82: Time spent starting tablet: real 0.017s user 0.004s sys 0.010s
I20260925 09:27:21.327119 18151 ts_tablet_manager.cc:1428] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106: Time spent starting tablet: real 0.020s user 0.017s sys 0.005s
I20260925 09:27:21.327998 18119 heartbeater.cc:503] Master 127.10.114.126:43419 was elected leader, sending a full tablet report...
I20260925 09:27:21.328645 17978 heartbeater.cc:503] Master 127.10.114.126:43419 was elected leader, sending a full tablet report...
I20260925 09:27:21.339697 18158 raft_consensus.cc:482] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:27:21.339977 18158 raft_consensus.cc:504] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 } } peers { permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 } } peers { permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 } }
I20260925 09:27:21.341423 18158 leader_election.cc:288] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd99001c7384aaab17cb23748b9611a (127.10.114.65:38265), 32afe2fd0d664cb588b6b4a2740e1106 (127.10.114.66:36841)
I20260925 09:27:21.346814 18146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.341769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:53909 (local address 127.10.114.65:38265)
0925 09:27:21.342074 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:21.342079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:21.342216 (+ 137us) server_negotiation.cc:407] Connection header received
0925 09:27:21.342386 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:21.342390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:21.342445 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:21.342534 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:21.342893 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.343555 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.345113 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.345840 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.346001 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.346065 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:21.346456 (+ 391us) server_negotiation.cc:299] Negotiation successful
0925 09:27:21.346588 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
I20260925 09:27:21.346833 18148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.341921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:46447 (local address 127.10.114.66:36841)
0925 09:27:21.342132 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:21.342136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:21.342149 (+ 13us) server_negotiation.cc:407] Connection header received
0925 09:27:21.342192 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:21.342195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:21.342241 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:21.342355 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:21.342711 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.343391 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.345024 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.345776 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.345937 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.346007 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:21.346476 (+ 469us) server_negotiation.cc:299] Negotiation successful
0925 09:27:21.346593 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:27:21.346870 18124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.341690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:36841 (local address 127.10.114.67:46447)
0925 09:27:21.341869 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:21.341885 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:21.341997 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:21.342371 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:21.342376 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:21.342400 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:21.342576 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:21.342584 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.343516 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.343522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:21.344875 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:21.344890 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.345937 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.345944 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:21.346262 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.346343 (+ 81us) client_negotiation.cc:769] Sending connection context
0925 09:27:21.346452 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 09:27:21.346585 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":63}
I20260925 09:27:21.346874 18125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.341781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:38265 (local address 127.10.114.67:53909)
0925 09:27:21.342103 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:21.342126 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:21.342244 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:21.342551 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:21.342555 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:21.342577 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:21.342771 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:21.342778 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.343703 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.343708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:21.344968 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:21.344979 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.346010 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.346015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:21.346269 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.346329 (+ 60us) client_negotiation.cc:769] Sending connection context
0925 09:27:21.346424 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 09:27:21.346559 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":20}
I20260925 09:27:21.347723 17932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb1cbb15930d4c54b3c61d9512a8e120" candidate_uuid: "2dbbfd760b8240e98b652fd145f7df82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32afe2fd0d664cb588b6b4a2740e1106" is_pre_election: true
I20260925 09:27:21.347997 17789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb1cbb15930d4c54b3c61d9512a8e120" candidate_uuid: "2dbbfd760b8240e98b652fd145f7df82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd99001c7384aaab17cb23748b9611a" is_pre_election: true
I20260925 09:27:21.348233 17932 raft_consensus.cc:2454] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2dbbfd760b8240e98b652fd145f7df82 in term 0.
I20260925 09:27:21.348429 17789 raft_consensus.cc:2454] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2dbbfd760b8240e98b652fd145f7df82 in term 0.
I20260925 09:27:21.348868 18009 leader_election.cc:302] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dbbfd760b8240e98b652fd145f7df82, 32afe2fd0d664cb588b6b4a2740e1106; no voters:
I20260925 09:27:21.349288 18158 raft_consensus.cc:2781] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:27:21.349428 18158 raft_consensus.cc:482] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:27:21.349519 18158 raft_consensus.cc:3037] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:21.351543 18158 raft_consensus.cc:504] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 } } peers { permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 } } peers { permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 } }
I20260925 09:27:21.352455 18158 leader_election.cc:288] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [CANDIDATE]: Term 1 election: Requested vote from peers 5cd99001c7384aaab17cb23748b9611a (127.10.114.65:38265), 32afe2fd0d664cb588b6b4a2740e1106 (127.10.114.66:36841)
I20260925 09:27:21.352804 17789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb1cbb15930d4c54b3c61d9512a8e120" candidate_uuid: "2dbbfd760b8240e98b652fd145f7df82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd99001c7384aaab17cb23748b9611a"
I20260925 09:27:21.352842 17932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb1cbb15930d4c54b3c61d9512a8e120" candidate_uuid: "2dbbfd760b8240e98b652fd145f7df82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32afe2fd0d664cb588b6b4a2740e1106"
I20260925 09:27:21.352978 17789 raft_consensus.cc:3037] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:21.352998 17932 raft_consensus.cc:3037] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:21.354949 17789 raft_consensus.cc:2454] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2dbbfd760b8240e98b652fd145f7df82 in term 1.
I20260925 09:27:21.355046 17932 raft_consensus.cc:2454] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2dbbfd760b8240e98b652fd145f7df82 in term 1.
I20260925 09:27:21.355355 18007 leader_election.cc:302] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dbbfd760b8240e98b652fd145f7df82, 5cd99001c7384aaab17cb23748b9611a; no voters:
I20260925 09:27:21.355620 18158 raft_consensus.cc:2781] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:21.356070 18158 raft_consensus.cc:686] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [term 1 LEADER]: Becoming Leader. State: Replica: 2dbbfd760b8240e98b652fd145f7df82, State: Running, Role: LEADER
I20260925 09:27:21.356555 18158 consensus_queue.cc:237] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 } } peers { permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 } } peers { permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 } }
I20260925 09:27:21.361611 16864 catalog_manager.cc:5495] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2dbbfd760b8240e98b652fd145f7df82 (127.10.114.67). New cstate: current_term: 1 leader_uuid: "2dbbfd760b8240e98b652fd145f7df82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 } health_report { overall_health: HEALTHY } } }
I20260925 09:27:21.394572 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.388825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51158 (local address 127.10.114.67:46759)
0925 09:27:21.389032 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:21.389037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:21.389055 (+ 18us) server_negotiation.cc:407] Connection header received
0925 09:27:21.389310 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:21.389325 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:21.389393 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:21.389535 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:27:21.389872 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.390984 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.392406 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.392861 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.393008 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.393470 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:27:21.394002 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:27:21.394161 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:21.394257 (+ 96us) server_negotiation.cc:299] Negotiation successful
0925 09:27:21.394328 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260925 09:27:21.397413 18121 tablet.cc:2369] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:27:21.402014 17932 raft_consensus.cc:1261] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106 [term 1 FOLLOWER]: Refusing update from remote peer 2dbbfd760b8240e98b652fd145f7df82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:27:21.402076 17789 raft_consensus.cc:1261] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a [term 1 FOLLOWER]: Refusing update from remote peer 2dbbfd760b8240e98b652fd145f7df82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:27:21.402912 18160 consensus_queue.cc:1035] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:27:21.403282 18158 consensus_queue.cc:1035] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:27:21.435588 16864 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:27:21.435880 16864 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:27:21.441789 16864 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de3ad4b7da3040669e2c44a8dfe9cf04: Sending DeleteTablet for 3 replicas of tablet fb1cbb15930d4c54b3c61d9512a8e120
I20260925 09:27:21.442771 17769 tablet_service.cc:1514] Processing DeleteTablet for tablet fb1cbb15930d4c54b3c61d9512a8e120 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:21 UTC) from {username='oryx', principal='oryx/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:40420
I20260925 09:27:21.443156 17912 tablet_service.cc:1514] Processing DeleteTablet for tablet fb1cbb15930d4c54b3c61d9512a8e120 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:21 UTC) from {username='oryx', principal='oryx/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:56362
I20260925 09:27:21.443993 18053 tablet_service.cc:1514] Processing DeleteTablet for tablet fb1cbb15930d4c54b3c61d9512a8e120 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:21 UTC) from {username='oryx', principal='oryx/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:51150
I20260925 09:27:21.444221 18174 tablet_replica.cc:330] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a: stopping tablet replica
I20260925 09:27:21.444617 18174 raft_consensus.cc:2229] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:27:21.445050 18174 raft_consensus.cc:2258] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:21.447532 18174 ts_tablet_manager.cc:1905] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:21.455087 18176 tablet_replica.cc:330] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82: stopping tablet replica
I20260925 09:27:21.455415 18176 raft_consensus.cc:2229] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:27:21.455915 18174 ts_tablet_manager.cc:1918] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:27:21.456065 18174 log.cc:1196] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/fb1cbb15930d4c54b3c61d9512a8e120
I20260925 09:27:21.456121 18176 raft_consensus.cc:2258] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:21.456629 18174 ts_tablet_manager.cc:1939] T fb1cbb15930d4c54b3c61d9512a8e120 P 5cd99001c7384aaab17cb23748b9611a: Deleting consensus metadata
I20260925 09:27:21.456892 18175 tablet_replica.cc:330] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106: stopping tablet replica
I20260925 09:27:21.457237 18175 raft_consensus.cc:2229] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:27:21.457639 18175 raft_consensus.cc:2258] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:21.458638 16850 catalog_manager.cc:4841] TS 5cd99001c7384aaab17cb23748b9611a (127.10.114.65:38265): tablet fb1cbb15930d4c54b3c61d9512a8e120 (table test-table [id=96fc52340c074489ae50f4846a4657ba]) successfully deleted
I20260925 09:27:21.459666 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 16833
I20260925 09:27:21.460060 18175 ts_tablet_manager.cc:1905] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:21.460784 16895 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:21.461525 18176 ts_tablet_manager.cc:1905] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:21.468578 18175 ts_tablet_manager.cc:1918] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:27:21.468739 18175 log.cc:1196] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/fb1cbb15930d4c54b3c61d9512a8e120
I20260925 09:27:21.469342 18175 ts_tablet_manager.cc:1939] T fb1cbb15930d4c54b3c61d9512a8e120 P 32afe2fd0d664cb588b6b4a2740e1106: Deleting consensus metadata
I20260925 09:27:21.476414 18176 ts_tablet_manager.cc:1918] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:27:21.476593 18176 log.cc:1196] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/fb1cbb15930d4c54b3c61d9512a8e120
I20260925 09:27:21.477229 18176 ts_tablet_manager.cc:1939] T fb1cbb15930d4c54b3c61d9512a8e120 P 2dbbfd760b8240e98b652fd145f7df82: Deleting consensus metadata
I20260925 09:27:21.482986 18122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.446746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34296 (local address 127.10.114.125:33507)
0925 09:27:21.446944 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:21.446951 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:21.458867 (+ 11916us) server_negotiation.cc:407] Connection header received
0925 09:27:21.461917 (+ 3050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:21.461923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:21.462003 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:21.462076 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:21.468335 (+ 6259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.469386 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.471232 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.472238 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.472360 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.476194 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:21.476200 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:21.476205 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:21.476239 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:21.479668 (+ 3429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:21.480711 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:21.480716 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:21.480720 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:21.480866 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:21.481326 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:21.481331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:21.481334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:21.481644 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:21.481877 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:21.482427 (+ 550us) server_negotiation.cc:299] Negotiation successful
0925 09:27:21.482643 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260925 09:27:21.499403 18127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.458976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43094 (local address 127.10.114.124:33261)
0925 09:27:21.459197 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:21.459203 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:21.483031 (+ 23828us) server_negotiation.cc:407] Connection header received
0925 09:27:21.483248 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:21.483254 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:21.483343 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:21.483410 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:21.485726 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.486831 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.488660 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.489275 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.489390 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.492451 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:21.492460 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:21.492464 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:21.492504 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:21.496250 (+ 3746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:21.497128 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:21.497135 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:21.497139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:21.497298 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:21.497689 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:21.497694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:21.497699 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:21.498098 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:21.498321 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:21.498684 (+ 363us) server_negotiation.cc:299] Negotiation successful
0925 09:27:21.498958 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260925 09:27:21.597570 16851 catalog_manager.cc:4841] TS 2dbbfd760b8240e98b652fd145f7df82 (127.10.114.67:46759): tablet fb1cbb15930d4c54b3c61d9512a8e120 (table test-table [id=96fc52340c074489ae50f4846a4657ba]) successfully deleted
I20260925 09:27:21.597734 16852 catalog_manager.cc:4841] TS 32afe2fd0d664cb588b6b4a2740e1106 (127.10.114.66:36841): tablet fb1cbb15930d4c54b3c61d9512a8e120 (table test-table [id=96fc52340c074489ae50f4846a4657ba]) successfully deleted
W20260925 09:27:21.597932 16852 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.114.66:36841, user_credentials={real_user=slave}} blocked reactor thread for 126692us
I20260925 09:27:21.598480 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 16833
I20260925 09:27:21.657636 18122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.633402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34310 (local address 127.10.114.125:33507)
0925 09:27:21.633622 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:21.633631 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:21.640584 (+ 6953us) server_negotiation.cc:407] Connection header received
0925 09:27:21.641085 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:21.641092 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:21.641184 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:21.641255 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:21.643461 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.644657 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.646699 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.647362 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.647459 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.651149 (+ 3690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:21.651156 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:21.651161 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:21.651197 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:21.654575 (+ 3378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:21.655373 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:21.655378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:21.655382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:21.655517 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:21.655972 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:21.655979 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:21.655985 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:21.656294 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:21.656500 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:21.657082 (+ 582us) server_negotiation.cc:299] Negotiation successful
0925 09:27:21.657301 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 09:27:21.674598 18127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:21.632973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43096 (local address 127.10.114.124:33261)
0925 09:27:21.633141 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:21.633148 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:21.658434 (+ 25286us) server_negotiation.cc:407] Connection header received
0925 09:27:21.658681 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:21.658687 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:21.658760 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:21.658833 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:21.660711 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.661852 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.663716 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:21.664423 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:21.664533 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:21.668511 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:21.668517 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:21.668522 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:21.668558 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:21.671891 (+ 3333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:21.672631 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:21.672636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:21.672640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:21.672786 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:21.673148 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:21.673151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:21.673153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:21.673426 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:21.673612 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:21.673982 (+ 370us) server_negotiation.cc:299] Negotiation successful
0925 09:27:21.674156 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:27:22.422969 18186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:22.422187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.66:49361)
0925 09:27:22.422678 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:22.422807 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":265,"threads_started":1}
W20260925 09:27:22.423460 17978 heartbeater.cc:644] Failed to heartbeat to 127.10.114.126:43419 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
I20260925 09:27:22.432264 18187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:22.431325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.67:41341)
0925 09:27:22.431759 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:22.431961 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":446,"threads_started":1}
W20260925 09:27:22.432698 18119 heartbeater.cc:644] Failed to heartbeat to 127.10.114.126:43419 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
I20260925 09:27:22.433171 18188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:22.432578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.65:39849)
0925 09:27:22.432962 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:22.433049 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":185,"threads_started":1}
W20260925 09:27:22.433521 17835 heartbeater.cc:644] Failed to heartbeat to 127.10.114.126:43419 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
I20260925 09:27:22.942832 18189 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de3ad4b7da3040669e2c44a8dfe9cf04)
I20260925 09:27:22.943115 18190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de3ad4b7da3040669e2c44a8dfe9cf04)
I20260925 09:27:22.943117 18189 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: false } }
I20260925 09:27:22.943341 18190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: false } }
I20260925 09:27:22.950713 18189 leader_election.cc:288] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de3ad4b7da3040669e2c44a8dfe9cf04 (127.10.114.126:43419), 0ad69ec73960431382f3b6085c0b0360 (127.10.114.125:33507)
I20260925 09:27:22.950899 18190 leader_election.cc:288] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de3ad4b7da3040669e2c44a8dfe9cf04 (127.10.114.126:43419), fe4bf2501e9144b79c313dc56a923f42 (127.10.114.124:33261)
I20260925 09:27:22.958902 18192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:22.944654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.0.0.1:53042)
0925 09:27:22.958243 (+ 13589us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:22.958465 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13446,"threads_started":1}
W20260925 09:27:22.960438 17640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
I20260925 09:27:22.960739 18193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:22.951213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.0.0.1:53052)
0925 09:27:22.960322 (+ 9109us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:22.960470 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8975,"thread_start_us":3645,"threads_started":1}
W20260925 09:27:22.961058 17456 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
I20260925 09:27:22.969517 18192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:22.944725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33507 (local address 127.0.0.1:34312)
0925 09:27:22.959279 (+ 14554us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:22.959300 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:22.959430 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:22.960767 (+ 1337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:22.960772 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:22.960797 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:22.961148 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:22.961162 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:22.963469 (+ 2307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:22.963477 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:22.966094 (+ 2617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:22.966114 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:22.968410 (+ 2296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:22.968419 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:22.968689 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:22.968743 (+ 54us) client_negotiation.cc:769] Sending connection context
0925 09:27:22.968823 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 09:27:22.969008 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14428,"mutex_wait_us":1,"thread_start_us":13216,"threads_started":1}
I20260925 09:27:22.970688 18195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:22.945018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34312 (local address 127.10.114.125:33507)
0925 09:27:22.958558 (+ 13540us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:22.958566 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:22.959543 (+ 977us) server_negotiation.cc:407] Connection header received
0925 09:27:22.959602 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:22.959606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:22.959678 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:22.959806 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:22.961426 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:22.963257 (+ 1831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:22.966329 (+ 3072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:22.968147 (+ 1818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:22.969861 (+ 1714us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:22.969967 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:22.970028 (+ 61us) server_negotiation.cc:299] Negotiation successful
0925 09:27:22.970209 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":120,"server-negotiator.queue_time_us":13404,"thread_start_us":13203,"threads_started":1}
I20260925 09:27:22.971297 17479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fe4bf2501e9144b79c313dc56a923f42" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0ad69ec73960431382f3b6085c0b0360" is_pre_election: true
I20260925 09:27:22.971627 17479 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe4bf2501e9144b79c313dc56a923f42 in term 1.
I20260925 09:27:22.972358 18194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:22.944939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43110 (local address 127.10.114.124:33261)
0925 09:27:22.959106 (+ 14167us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:22.959114 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:22.959139 (+ 25us) server_negotiation.cc:407] Connection header received
0925 09:27:22.959200 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:22.959208 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:22.959316 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:22.959442 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:22.960061 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:22.961346 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:22.964872 (+ 3526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:22.971081 (+ 6209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:22.971223 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:22.971299 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:22.971867 (+ 568us) server_negotiation.cc:299] Negotiation successful
0925 09:27:22.971996 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14016,"thread_start_us":13847,"threads_started":1}
I20260925 09:27:22.972316 17639 leader_election.cc:302] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad69ec73960431382f3b6085c0b0360, fe4bf2501e9144b79c313dc56a923f42; no voters:
I20260925 09:27:22.972688 18189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 09:27:22.972796 18189 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 1 FOLLOWER]: Starting leader election (detected failure of leader de3ad4b7da3040669e2c44a8dfe9cf04)
I20260925 09:27:22.972878 18189 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:27:22.973065 18196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:22.944785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:33261 (local address 127.0.0.1:43110)
0925 09:27:22.955126 (+ 10341us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:22.955151 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:22.955328 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:22.959475 (+ 4147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:22.959482 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:22.959512 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:22.959825 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:22.959837 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:22.961537 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:22.961545 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:22.964683 (+ 3138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:22.964700 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:22.971462 (+ 6762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:22.971470 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:22.971658 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:22.971689 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 09:27:22.972497 (+ 808us) client_negotiation.cc:240] Negotiation successful
0925 09:27:22.972626 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10205,"mutex_wait_us":132,"thread_start_us":15441,"threads_started":1}
I20260925 09:27:22.973623 17663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ad69ec73960431382f3b6085c0b0360" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fe4bf2501e9144b79c313dc56a923f42" is_pre_election: true
I20260925 09:27:22.974027 18196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:22.973646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.0.0.1:53062)
0925 09:27:22.973830 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:22.973904 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260925 09:27:22.974264 17456 leader_election.cc:334] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer de3ad4b7da3040669e2c44a8dfe9cf04 (127.10.114.126:43419): Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
I20260925 09:27:22.974997 18192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:22.974642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.0.0.1:53076)
0925 09:27:22.974772 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:22.974856 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 09:27:22.975241 17640 leader_election.cc:334] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer de3ad4b7da3040669e2c44a8dfe9cf04 (127.10.114.126:43419): Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
I20260925 09:27:22.975965 18189 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: false } }
I20260925 09:27:22.976351 17663 raft_consensus.cc:2379] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ad69ec73960431382f3b6085c0b0360 in current term 2: Already voted for candidate fe4bf2501e9144b79c313dc56a923f42 in this term.
I20260925 09:27:22.976588 18189 leader_election.cc:288] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [CANDIDATE]: Term 2 election: Requested vote from peers de3ad4b7da3040669e2c44a8dfe9cf04 (127.10.114.126:43419), 0ad69ec73960431382f3b6085c0b0360 (127.10.114.125:33507)
I20260925 09:27:22.977023 18192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:22.976619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.0.0.1:53078)
0925 09:27:22.976809 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:22.976865 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260925 09:27:22.977023 17479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fe4bf2501e9144b79c313dc56a923f42" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0ad69ec73960431382f3b6085c0b0360"
I20260925 09:27:22.977223 17479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:27:22.977998 18192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:22.977673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.0.0.1:53082)
0925 09:27:22.977817 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:22.977883 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260925 09:27:22.978214 17640 leader_election.cc:334] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer de3ad4b7da3040669e2c44a8dfe9cf04 (127.10.114.126:43419): Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
I20260925 09:27:22.979610 17479 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe4bf2501e9144b79c313dc56a923f42 in term 2.
I20260925 09:27:22.979961 17639 leader_election.cc:302] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ad69ec73960431382f3b6085c0b0360, fe4bf2501e9144b79c313dc56a923f42; no voters: de3ad4b7da3040669e2c44a8dfe9cf04
I20260925 09:27:22.979955 17453 leader_election.cc:302] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ad69ec73960431382f3b6085c0b0360; no voters: de3ad4b7da3040669e2c44a8dfe9cf04, fe4bf2501e9144b79c313dc56a923f42
I20260925 09:27:22.980290 18190 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 09:27:22.980322 18189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:27:22.980711 18189 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [term 2 LEADER]: Becoming Leader. State: Replica: fe4bf2501e9144b79c313dc56a923f42, State: Running, Role: LEADER
I20260925 09:27:22.981060 18189 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: false } }
I20260925 09:27:22.982805 18199 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe4bf2501e9144b79c313dc56a923f42. Latest consensus state: current_term: 2 leader_uuid: "fe4bf2501e9144b79c313dc56a923f42" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: false } } }
I20260925 09:27:22.983141 18199 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:22.983832 18201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:27:22.988009 18201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:27:22.988801 18201 catalog_manager.cc:1259] Loaded cluster ID: 5344fdedc2704f5782bd1313986aaf4e
I20260925 09:27:22.988865 18201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:27:22.989887 18201 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:27:22.990576 18201 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: Loaded TSK: 0
I20260925 09:27:22.991317 18201 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:27:23.043849 17652 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:27:23.050953 17652 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:27:23.054570 18192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.054219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.0.0.1:53110)
0925 09:27:23.054362 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.054430 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260925 09:27:23.054754 17479 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [term 2 FOLLOWER]: Refusing update from remote peer fe4bf2501e9144b79c313dc56a923f42: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260925 09:27:23.055572 18192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.055162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.0.0.1:53114)
0925 09:27:23.055332 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.055456 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 09:27:23.055539 18199 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260925 09:27:23.055859 17640 consensus_peers.cc:487] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 -> Peer de3ad4b7da3040669e2c44a8dfe9cf04 (127.10.114.126:43419): Couldn't send request to peer de3ad4b7da3040669e2c44a8dfe9cf04. Status: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 09:27:23.057890 18190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe4bf2501e9144b79c313dc56a923f42. Latest consensus state: current_term: 2 leader_uuid: "fe4bf2501e9144b79c313dc56a923f42" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: false } } }
I20260925 09:27:23.058111 18190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:27:23.059671 18189 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "fe4bf2501e9144b79c313dc56a923f42" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: false } } }
I20260925 09:27:23.060036 18189 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:23.060139 18190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "fe4bf2501e9144b79c313dc56a923f42" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "de3ad4b7da3040669e2c44a8dfe9cf04" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43419 } } peers { permanent_uuid: "0ad69ec73960431382f3b6085c0b0360" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33507 } attrs { promote: false } } peers { permanent_uuid: "fe4bf2501e9144b79c313dc56a923f42" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 33261 } attrs { promote: false } } }
I20260925 09:27:23.060366 18190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ad69ec73960431382f3b6085c0b0360 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:27:23.060475 18203 mvcc.cc:205] Tried to move back new op lower bound from 7333185302745960448 to 7333185302453727232. Current Snapshot: MvccSnapshot[applied={T|T < 7333185302745960448}]
I20260925 09:27:23.085846 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.067450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:38265 (local address 127.0.0.1:40428)
0925 09:27:23.067855 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.067872 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:23.067969 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:23.078845 (+ 10876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:23.078853 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:23.078902 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:23.079435 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:23.079450 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.081276 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.081282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:23.083477 (+ 2195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:23.083490 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.085078 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.085084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:23.085279 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:23.085311 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 09:27:23.085391 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 09:27:23.085513 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":16}
I20260925 09:27:23.085851 18211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.068207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40428 (local address 127.10.114.65:38265)
0925 09:27:23.078331 (+ 10124us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:23.078340 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:23.078365 (+ 25us) server_negotiation.cc:407] Connection header received
0925 09:27:23.078476 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:23.078484 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:23.078582 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:23.079670 (+ 1088us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:23.079893 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.081105 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.083676 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.084843 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.085274 (+ 431us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:23.085383 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:23.085428 (+ 45us) server_negotiation.cc:299] Negotiation successful
0925 09:27:23.085545 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9948,"thread_start_us":10445,"threads_started":1}
I20260925 09:27:23.086685 17769 tablet_service.cc:1467] Processing CreateTablet for tablet f2b319c66c3d497e8ebdc53bae0d86a8 (DEFAULT_TABLE table=test-table [id=8e2a5ef9894f4e5bbf0af77c97a692b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:23.087167 17769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2b319c66c3d497e8ebdc53bae0d86a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:23.089003 18209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.067915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:46759 (local address 127.0.0.1:51160)
0925 09:27:23.081272 (+ 13357us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.081287 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:23.081488 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:23.082036 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:23.082040 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:23.082061 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:23.082402 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:23.082412 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.083861 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.083866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:23.086903 (+ 3037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:23.086915 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.088181 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.088187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:23.088382 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:23.088411 (+ 29us) client_negotiation.cc:769] Sending connection context
0925 09:27:23.088492 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 09:27:23.088622 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13215,"mutex_wait_us":22,"thread_start_us":13099,"threads_started":1}
I20260925 09:27:23.089582 18210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.068008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51160 (local address 127.10.114.67:46759)
0925 09:27:23.080892 (+ 12884us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:23.080897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:23.081383 (+ 486us) server_negotiation.cc:407] Connection header received
0925 09:27:23.081645 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:23.081651 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:23.081774 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:23.081845 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:23.082583 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.083650 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.087019 (+ 3369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.087762 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.087924 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:23.088041 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:23.089256 (+ 1215us) server_negotiation.cc:299] Negotiation successful
0925 09:27:23.089390 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12787,"thread_start_us":12640,"threads_started":1}
I20260925 09:27:23.090291 18053 tablet_service.cc:1467] Processing CreateTablet for tablet f2b319c66c3d497e8ebdc53bae0d86a8 (DEFAULT_TABLE table=test-table [id=8e2a5ef9894f4e5bbf0af77c97a692b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:23.090683 18053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2b319c66c3d497e8ebdc53bae0d86a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:23.092595 18192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.067054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:36841 (local address 127.0.0.1:56364)
0925 09:27:23.067261 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.067281 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:23.067406 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:23.086710 (+ 19304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:23.086717 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:23.086749 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:23.086939 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:23.086951 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.088350 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.088356 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:23.090249 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:23.090264 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.091907 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.091912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:23.092114 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:23.092154 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 09:27:23.092237 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 09:27:23.092368 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260925 09:27:23.092945 18208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.067152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56364 (local address 127.10.114.66:36841)
0925 09:27:23.086359 (+ 19207us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:23.086365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:23.086383 (+ 18us) server_negotiation.cc:407] Connection header received
0925 09:27:23.086443 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:23.086448 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:23.086518 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:23.086622 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:23.087128 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.088177 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.090488 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.091703 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.091906 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:23.092013 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:23.092296 (+ 283us) server_negotiation.cc:299] Negotiation successful
0925 09:27:23.092449 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19072,"thread_start_us":18981,"threads_started":1}
I20260925 09:27:23.093680 17912 tablet_service.cc:1467] Processing CreateTablet for tablet f2b319c66c3d497e8ebdc53bae0d86a8 (DEFAULT_TABLE table=test-table [id=8e2a5ef9894f4e5bbf0af77c97a692b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:23.094215 17912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2b319c66c3d497e8ebdc53bae0d86a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:23.096024 18213 tablet_bootstrap.cc:492] T f2b319c66c3d497e8ebdc53bae0d86a8 P 2dbbfd760b8240e98b652fd145f7df82: Bootstrap starting.
I20260925 09:27:23.097738 18213 tablet_bootstrap.cc:654] T f2b319c66c3d497e8ebdc53bae0d86a8 P 2dbbfd760b8240e98b652fd145f7df82: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:23.102072 18212 tablet_bootstrap.cc:492] T f2b319c66c3d497e8ebdc53bae0d86a8 P 5cd99001c7384aaab17cb23748b9611a: Bootstrap starting.
I20260925 09:27:23.104555 18212 tablet_bootstrap.cc:654] T f2b319c66c3d497e8ebdc53bae0d86a8 P 5cd99001c7384aaab17cb23748b9611a: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:23.105283 18215 tablet_bootstrap.cc:492] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106: Bootstrap starting.
I20260925 09:27:23.106878 18212 tablet_bootstrap.cc:492] T f2b319c66c3d497e8ebdc53bae0d86a8 P 5cd99001c7384aaab17cb23748b9611a: No bootstrap required, opened a new log
I20260925 09:27:23.107031 18212 ts_tablet_manager.cc:1397] T f2b319c66c3d497e8ebdc53bae0d86a8 P 5cd99001c7384aaab17cb23748b9611a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 09:27:23.108366 18215 tablet_bootstrap.cc:654] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:23.108310 18212 raft_consensus.cc:348] T f2b319c66c3d497e8ebdc53bae0d86a8 P 5cd99001c7384aaab17cb23748b9611a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 } } peers { permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 } } peers { permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 } }
I20260925 09:27:23.108707 18212 raft_consensus.cc:374] T f2b319c66c3d497e8ebdc53bae0d86a8 P 5cd99001c7384aaab17cb23748b9611a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:23.108805 18212 raft_consensus.cc:729] T f2b319c66c3d497e8ebdc53bae0d86a8 P 5cd99001c7384aaab17cb23748b9611a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd99001c7384aaab17cb23748b9611a, State: Initialized, Role: FOLLOWER
I20260925 09:27:23.109155 18212 consensus_queue.cc:260] T f2b319c66c3d497e8ebdc53bae0d86a8 P 5cd99001c7384aaab17cb23748b9611a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 } } peers { permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 } } peers { permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 } }
I20260925 09:27:23.109839 18213 tablet_bootstrap.cc:492] T f2b319c66c3d497e8ebdc53bae0d86a8 P 2dbbfd760b8240e98b652fd145f7df82: No bootstrap required, opened a new log
I20260925 09:27:23.110013 18213 ts_tablet_manager.cc:1397] T f2b319c66c3d497e8ebdc53bae0d86a8 P 2dbbfd760b8240e98b652fd145f7df82: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.003s
I20260925 09:27:23.110049 18212 ts_tablet_manager.cc:1428] T f2b319c66c3d497e8ebdc53bae0d86a8 P 5cd99001c7384aaab17cb23748b9611a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 09:27:23.110953 18213 raft_consensus.cc:348] T f2b319c66c3d497e8ebdc53bae0d86a8 P 2dbbfd760b8240e98b652fd145f7df82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 } } peers { permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 } } peers { permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 } }
I20260925 09:27:23.111315 18213 raft_consensus.cc:374] T f2b319c66c3d497e8ebdc53bae0d86a8 P 2dbbfd760b8240e98b652fd145f7df82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:23.111452 18213 raft_consensus.cc:729] T f2b319c66c3d497e8ebdc53bae0d86a8 P 2dbbfd760b8240e98b652fd145f7df82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dbbfd760b8240e98b652fd145f7df82, State: Initialized, Role: FOLLOWER
I20260925 09:27:23.111788 18213 consensus_queue.cc:260] T f2b319c66c3d497e8ebdc53bae0d86a8 P 2dbbfd760b8240e98b652fd145f7df82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 } } peers { permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 } } peers { permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 } }
I20260925 09:27:23.113834 17834 heartbeater.cc:503] Master 127.10.114.124:33261 was elected leader, sending a full tablet report...
I20260925 09:27:23.117637 18219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.110617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.65:37849)
0925 09:27:23.117119 (+ 6502us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.117333 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6344,"threads_started":1}
I20260925 09:27:23.118537 18213 ts_tablet_manager.cc:1428] T f2b319c66c3d497e8ebdc53bae0d86a8 P 2dbbfd760b8240e98b652fd145f7df82: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20260925 09:27:23.119961 18221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.119042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.67:36483)
0925 09:27:23.119511 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.119677 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"threads_started":1}
I20260925 09:27:23.120360 18118 heartbeater.cc:503] Master 127.10.114.124:33261 was elected leader, sending a full tablet report...
I20260925 09:27:23.120823 18219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.120474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.65:53781)
0925 09:27:23.120618 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.120697 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260925 09:27:23.121387 18215 tablet_bootstrap.cc:492] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106: No bootstrap required, opened a new log
I20260925 09:27:23.121521 18215 ts_tablet_manager.cc:1397] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.010s
I20260925 09:27:23.122215 18215 raft_consensus.cc:348] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 } } peers { permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 } } peers { permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 } }
I20260925 09:27:23.122385 18215 raft_consensus.cc:374] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:23.122449 18215 raft_consensus.cc:729] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32afe2fd0d664cb588b6b4a2740e1106, State: Initialized, Role: FOLLOWER
I20260925 09:27:23.122655 18215 consensus_queue.cc:260] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 } } peers { permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 } } peers { permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 } }
I20260925 09:27:23.123421 18215 ts_tablet_manager.cc:1428] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 09:27:23.124487 18221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.123427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.67:52889)
0925 09:27:23.123905 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.123989 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382}
I20260925 09:27:23.125464 18223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.124032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.66:44259)
0925 09:27:23.125093 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.125250 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":956,"thread_start_us":825,"threads_started":1}
I20260925 09:27:23.126399 18223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.126003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.66:49961)
0925 09:27:23.126186 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.126261 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260925 09:27:23.126425 17977 heartbeater.cc:503] Master 127.10.114.124:33261 was elected leader, sending a full tablet report...
I20260925 09:27:23.145503 18222 raft_consensus.cc:482] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:27:23.145725 18222 raft_consensus.cc:504] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 } } peers { permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 } } peers { permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 } }
I20260925 09:27:23.147446 18222 leader_election.cc:288] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd99001c7384aaab17cb23748b9611a (127.10.114.65:38265), 2dbbfd760b8240e98b652fd145f7df82 (127.10.114.67:46759)
I20260925 09:27:23.153978 18211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.147827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:39847 (local address 127.10.114.65:38265)
0925 09:27:23.148104 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:23.148109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:23.148661 (+ 552us) server_negotiation.cc:407] Connection header received
0925 09:27:23.148934 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:23.148940 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:23.149016 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:23.149106 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:23.149671 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.150340 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.152022 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.153110 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.153269 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:23.153334 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:23.153613 (+ 279us) server_negotiation.cc:299] Negotiation successful
0925 09:27:23.153738 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260925 09:27:23.154273 18210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.147882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:53289 (local address 127.10.114.67:46759)
0925 09:27:23.148067 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:23.148073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:23.148092 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:27:23.148148 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:23.148153 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:23.148215 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:23.148295 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:23.148875 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.149882 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.152264 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.153417 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.153590 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:23.153655 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:23.153957 (+ 302us) server_negotiation.cc:299] Negotiation successful
0925 09:27:23.154074 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 09:27:23.154302 18223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.147569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:46759 (local address 127.10.114.66:53289)
0925 09:27:23.147861 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.147878 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:23.148009 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:23.148447 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:23.148452 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:23.148477 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:23.148740 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:23.148749 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.150041 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.150046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:23.152123 (+ 2077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:23.152136 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.153620 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.153625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:23.153782 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:23.153802 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 09:27:23.153912 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 09:27:23.154041 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260925 09:27:23.154736 18224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.147569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:38265 (local address 127.10.114.66:39847)
0925 09:27:23.148581 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.148599 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:23.148780 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:23.149281 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:23.149286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:23.149311 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:23.149535 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:23.149542 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.150474 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.150480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:23.151861 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:23.151872 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.153276 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.153281 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:23.153443 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:23.153473 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 09:27:23.154190 (+ 717us) client_negotiation.cc:240] Negotiation successful
0925 09:27:23.154296 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"mutex_wait_us":551,"thread_start_us":681,"threads_started":1}
I20260925 09:27:23.155234 18073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2b319c66c3d497e8ebdc53bae0d86a8" candidate_uuid: "32afe2fd0d664cb588b6b4a2740e1106" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dbbfd760b8240e98b652fd145f7df82" is_pre_election: true
I20260925 09:27:23.155534 18073 raft_consensus.cc:2454] T f2b319c66c3d497e8ebdc53bae0d86a8 P 2dbbfd760b8240e98b652fd145f7df82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32afe2fd0d664cb588b6b4a2740e1106 in term 0.
I20260925 09:27:23.156036 17867 leader_election.cc:302] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dbbfd760b8240e98b652fd145f7df82, 32afe2fd0d664cb588b6b4a2740e1106; no voters:
I20260925 09:27:23.156155 17789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2b319c66c3d497e8ebdc53bae0d86a8" candidate_uuid: "32afe2fd0d664cb588b6b4a2740e1106" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd99001c7384aaab17cb23748b9611a" is_pre_election: true
I20260925 09:27:23.156402 17789 raft_consensus.cc:2454] T f2b319c66c3d497e8ebdc53bae0d86a8 P 5cd99001c7384aaab17cb23748b9611a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32afe2fd0d664cb588b6b4a2740e1106 in term 0.
I20260925 09:27:23.156421 18222 raft_consensus.cc:2781] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:27:23.156544 18222 raft_consensus.cc:482] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:27:23.156665 18222 raft_consensus.cc:3037] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:23.158757 18222 raft_consensus.cc:504] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 } } peers { permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 } } peers { permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 } }
I20260925 09:27:23.159283 18222 leader_election.cc:288] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [CANDIDATE]: Term 1 election: Requested vote from peers 5cd99001c7384aaab17cb23748b9611a (127.10.114.65:38265), 2dbbfd760b8240e98b652fd145f7df82 (127.10.114.67:46759)
I20260925 09:27:23.159610 17789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2b319c66c3d497e8ebdc53bae0d86a8" candidate_uuid: "32afe2fd0d664cb588b6b4a2740e1106" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd99001c7384aaab17cb23748b9611a"
I20260925 09:27:23.159718 18073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2b319c66c3d497e8ebdc53bae0d86a8" candidate_uuid: "32afe2fd0d664cb588b6b4a2740e1106" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dbbfd760b8240e98b652fd145f7df82"
I20260925 09:27:23.159825 17789 raft_consensus.cc:3037] T f2b319c66c3d497e8ebdc53bae0d86a8 P 5cd99001c7384aaab17cb23748b9611a [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:23.159969 18073 raft_consensus.cc:3037] T f2b319c66c3d497e8ebdc53bae0d86a8 P 2dbbfd760b8240e98b652fd145f7df82 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:23.161754 17789 raft_consensus.cc:2454] T f2b319c66c3d497e8ebdc53bae0d86a8 P 5cd99001c7384aaab17cb23748b9611a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32afe2fd0d664cb588b6b4a2740e1106 in term 1.
I20260925 09:27:23.162019 18073 raft_consensus.cc:2454] T f2b319c66c3d497e8ebdc53bae0d86a8 P 2dbbfd760b8240e98b652fd145f7df82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32afe2fd0d664cb588b6b4a2740e1106 in term 1.
I20260925 09:27:23.162178 17866 leader_election.cc:302] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32afe2fd0d664cb588b6b4a2740e1106, 5cd99001c7384aaab17cb23748b9611a; no voters:
I20260925 09:27:23.162492 18222 raft_consensus.cc:2781] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:23.162961 18222 raft_consensus.cc:686] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [term 1 LEADER]: Becoming Leader. State: Replica: 32afe2fd0d664cb588b6b4a2740e1106, State: Running, Role: LEADER
I20260925 09:27:23.163339 18222 consensus_queue.cc:237] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 } } peers { permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 } } peers { permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 } }
I20260925 09:27:23.163859 18224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.163464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.66:35341)
0925 09:27:23.163601 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.163705 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 09:27:23.166580 17652 catalog_manager.cc:5495] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32afe2fd0d664cb588b6b4a2740e1106 (127.10.114.66). New cstate: current_term: 1 leader_uuid: "32afe2fd0d664cb588b6b4a2740e1106" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32afe2fd0d664cb588b6b4a2740e1106" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 36841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 } health_report { overall_health: UNKNOWN } } }
I20260925 09:27:23.193611 18208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.189069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56368 (local address 127.10.114.66:36841)
0925 09:27:23.189227 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:23.189231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:23.189273 (+ 42us) server_negotiation.cc:407] Connection header received
0925 09:27:23.189447 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:23.189450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:23.189519 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:23.189627 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:27:23.189898 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.190499 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.191624 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.192084 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.192172 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:23.192515 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:27:23.193044 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:27:23.193137 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:23.193275 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 09:27:23.193343 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 09:27:23.197657 17789 raft_consensus.cc:1261] T f2b319c66c3d497e8ebdc53bae0d86a8 P 5cd99001c7384aaab17cb23748b9611a [term 1 FOLLOWER]: Refusing update from remote peer 32afe2fd0d664cb588b6b4a2740e1106: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:27:23.197968 18073 raft_consensus.cc:1261] T f2b319c66c3d497e8ebdc53bae0d86a8 P 2dbbfd760b8240e98b652fd145f7df82 [term 1 FOLLOWER]: Refusing update from remote peer 32afe2fd0d664cb588b6b4a2740e1106: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:27:23.198410 18225 consensus_queue.cc:1035] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd99001c7384aaab17cb23748b9611a" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:27:23.198801 18222 consensus_queue.cc:1035] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dbbfd760b8240e98b652fd145f7df82" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:27:23.201426 18219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.200882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.65:57505)
0925 09:27:23.201117 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.201233 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260925 09:27:23.201821 18221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.201238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.67:39281)
0925 09:27:23.201489 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.201596 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260925 09:27:23.211289 18230 mvcc.cc:205] Tried to move back new op lower bound from 7333185303330627584 to 7333185303201595392. Current Snapshot: MvccSnapshot[applied={T|T < 7333185303330627584}]
I20260925 09:27:23.212316 18224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.211702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.66:53193)
0925 09:27:23.212016 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.212138 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206}
I20260925 09:27:23.212844 18219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.212223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.65:60997)
0925 09:27:23.212594 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.212688 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265}
I20260925 09:27:23.212925 18221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.212476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.10.114.67:45469)
0925 09:27:23.212668 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.212767 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260925 09:27:23.225822 17653 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:27:23.226112 17653 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:27:23.231738 17653 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fe4bf2501e9144b79c313dc56a923f42: Sending DeleteTablet for 3 replicas of tablet f2b319c66c3d497e8ebdc53bae0d86a8
I20260925 09:27:23.232610 17912 tablet_service.cc:1514] Processing DeleteTablet for tablet f2b319c66c3d497e8ebdc53bae0d86a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:23 UTC) from {username='oryx', principal='oryx/127.10.114.124@KRBTEST.COM'} at 127.0.0.1:56364
I20260925 09:27:23.232945 18053 tablet_service.cc:1514] Processing DeleteTablet for tablet f2b319c66c3d497e8ebdc53bae0d86a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:23 UTC) from {username='oryx', principal='oryx/127.10.114.124@KRBTEST.COM'} at 127.0.0.1:51160
I20260925 09:27:23.233023 17769 tablet_service.cc:1514] Processing DeleteTablet for tablet f2b319c66c3d497e8ebdc53bae0d86a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:23 UTC) from {username='oryx', principal='oryx/127.10.114.124@KRBTEST.COM'} at 127.0.0.1:40428
I20260925 09:27:23.238677 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 17625
I20260925 09:27:23.246726 18239 tablet_replica.cc:330] T f2b319c66c3d497e8ebdc53bae0d86a8 P 2dbbfd760b8240e98b652fd145f7df82: stopping tablet replica
I20260925 09:27:23.247071 18239 raft_consensus.cc:2229] T f2b319c66c3d497e8ebdc53bae0d86a8 P 2dbbfd760b8240e98b652fd145f7df82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:27:23.247429 18239 raft_consensus.cc:2258] T f2b319c66c3d497e8ebdc53bae0d86a8 P 2dbbfd760b8240e98b652fd145f7df82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:23.248873 18239 ts_tablet_manager.cc:1905] T f2b319c66c3d497e8ebdc53bae0d86a8 P 2dbbfd760b8240e98b652fd145f7df82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:23.255088 18238 tablet_replica.cc:330] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106: stopping tablet replica
I20260925 09:27:23.256390 18239 ts_tablet_manager.cc:1918] T f2b319c66c3d497e8ebdc53bae0d86a8 P 2dbbfd760b8240e98b652fd145f7df82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:27:23.256559 18239 log.cc:1196] T f2b319c66c3d497e8ebdc53bae0d86a8 P 2dbbfd760b8240e98b652fd145f7df82: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/f2b319c66c3d497e8ebdc53bae0d86a8
I20260925 09:27:23.257067 18239 ts_tablet_manager.cc:1939] T f2b319c66c3d497e8ebdc53bae0d86a8 P 2dbbfd760b8240e98b652fd145f7df82: Deleting consensus metadata
I20260925 09:27:23.257421 18240 tablet_replica.cc:330] T f2b319c66c3d497e8ebdc53bae0d86a8 P 5cd99001c7384aaab17cb23748b9611a: stopping tablet replica
I20260925 09:27:23.257752 18238 raft_consensus.cc:2229] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:27:23.258054 18240 raft_consensus.cc:2229] T f2b319c66c3d497e8ebdc53bae0d86a8 P 5cd99001c7384aaab17cb23748b9611a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:27:23.258270 17639 catalog_manager.cc:4841] TS 2dbbfd760b8240e98b652fd145f7df82 (127.10.114.67:46759): tablet f2b319c66c3d497e8ebdc53bae0d86a8 (table test-table [id=8e2a5ef9894f4e5bbf0af77c97a692b5]) successfully deleted
I20260925 09:27:23.258455 18240 raft_consensus.cc:2258] T f2b319c66c3d497e8ebdc53bae0d86a8 P 5cd99001c7384aaab17cb23748b9611a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:23.259220 18238 raft_consensus.cc:2258] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:23.260484 18240 ts_tablet_manager.cc:1905] T f2b319c66c3d497e8ebdc53bae0d86a8 P 5cd99001c7384aaab17cb23748b9611a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:23.261412 18238 ts_tablet_manager.cc:1905] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:23.267659 18240 ts_tablet_manager.cc:1918] T f2b319c66c3d497e8ebdc53bae0d86a8 P 5cd99001c7384aaab17cb23748b9611a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:27:23.267879 18240 log.cc:1196] T f2b319c66c3d497e8ebdc53bae0d86a8 P 5cd99001c7384aaab17cb23748b9611a: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/f2b319c66c3d497e8ebdc53bae0d86a8
I20260925 09:27:23.268256 18238 ts_tablet_manager.cc:1918] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:27:23.268399 18240 ts_tablet_manager.cc:1939] T f2b319c66c3d497e8ebdc53bae0d86a8 P 5cd99001c7384aaab17cb23748b9611a: Deleting consensus metadata
I20260925 09:27:23.268433 18238 log.cc:1196] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/f2b319c66c3d497e8ebdc53bae0d86a8
I20260925 09:27:23.268999 18238 ts_tablet_manager.cc:1939] T f2b319c66c3d497e8ebdc53bae0d86a8 P 32afe2fd0d664cb588b6b4a2740e1106: Deleting consensus metadata
I20260925 09:27:23.269570 17638 catalog_manager.cc:4841] TS 5cd99001c7384aaab17cb23748b9611a (127.10.114.65:38265): tablet f2b319c66c3d497e8ebdc53bae0d86a8 (table test-table [id=8e2a5ef9894f4e5bbf0af77c97a692b5]) successfully deleted
I20260925 09:27:23.270031 17640 catalog_manager.cc:4841] TS 32afe2fd0d664cb588b6b4a2740e1106 (127.10.114.66:36841): tablet f2b319c66c3d497e8ebdc53bae0d86a8 (table test-table [id=8e2a5ef9894f4e5bbf0af77c97a692b5]) successfully deleted
I20260925 09:27:23.272315 18211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.239936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40442 (local address 127.10.114.65:38265)
0925 09:27:23.240110 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:23.240119 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:23.254383 (+ 14264us) server_negotiation.cc:407] Connection header received
0925 09:27:23.254559 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:23.254564 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:23.254646 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:23.254753 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:23.256989 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.258259 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.260282 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.260946 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.261078 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:23.264808 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:23.264824 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:23.264844 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:23.264893 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:23.269025 (+ 4132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:23.270225 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:23.270236 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:23.270240 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:23.270362 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:23.270741 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:23.270746 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:23.270749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:23.271111 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:23.271427 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:23.271731 (+ 304us) server_negotiation.cc:299] Negotiation successful
0925 09:27:23.271994 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 09:27:23.272888 17829 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:23.367635 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 17625
I20260925 09:27:23.385295 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 17845
I20260925 09:27:23.401044 18208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.386070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56376 (local address 127.10.114.66:36841)
0925 09:27:23.386273 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:23.386280 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:23.386332 (+ 52us) server_negotiation.cc:407] Connection header received
0925 09:27:23.386504 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:23.386510 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:23.386604 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:23.386726 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:23.387998 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.389235 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.390448 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.391162 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.391301 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:23.394143 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:23.394168 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:23.394190 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:23.394226 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:23.398059 (+ 3833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:23.398719 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:23.398735 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:23.398739 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:23.398930 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:23.399281 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:23.399286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:23.399289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:23.399735 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:23.400074 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:23.400399 (+ 325us) server_negotiation.cc:299] Negotiation successful
0925 09:27:23.400621 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 09:27:23.401686 17972 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:23.466147 18192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.465735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.0.0.1:37738)
0925 09:27:23.465901 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.465996 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260925 09:27:23.467094 18192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.466763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43419 (local address 127.0.0.1:37748)
0925 09:27:23.466933 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:23.466995 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 09:27:23.518044 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 17845
I20260925 09:27:23.537237 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 17990
I20260925 09:27:23.549602 18210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:23.537953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41440 (local address 127.10.114.67:46759)
0925 09:27:23.538105 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:23.538109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:23.538214 (+ 105us) server_negotiation.cc:407] Connection header received
0925 09:27:23.538348 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:23.538352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:23.538409 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:23.538486 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:23.539812 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.540519 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.541625 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:23.541980 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:23.542067 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:23.544601 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:23.544614 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:23.544631 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:23.544672 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:23.547296 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:23.547939 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:23.547951 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:23.547956 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:23.548105 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:23.548406 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:23.548409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:23.548411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:23.548774 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:23.548997 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:23.549208 (+ 211us) server_negotiation.cc:299] Negotiation successful
0925 09:27:23.549353 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 09:27:23.550220 18113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:23.665707 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 17990
I20260925 09:27:23.687074 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 17326
I20260925 09:27:23.688130 17499 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:23.807401 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 17326
I20260925 09:27:23.828320 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 17427
I20260925 09:27:23.829293 17683 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:23.944418 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 17427
2026-09-25T09:27:23Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9011 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:27:24 dist-test-slave-9pj8 krb5kdc[18249](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:27:24 dist-test-slave-9pj8 krb5kdc[18249](info): set up 2 sockets
Sep 25 09:27:24 dist-test-slave-9pj8 krb5kdc[18249](info): commencing operation
krb5kdc: starting...
W20260925 09:27:25.032716 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:27:25 dist-test-slave-9pj8 krb5kdc[18249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:27:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:27:25Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.126.keytab.
Entry for principal oryx/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.126.keytab.
I20260925 09:27:25.223559 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:44883
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.126.keytab
--principal=oryx/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:44883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:25.535094 18262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:25.535429 18262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:25.535595 18262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:25.542464 18262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:27:25.542620 18262 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 09:27:25.542716 18262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:25.542833 18262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:25.542924 18262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:27:25.543017 18262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:27:25.552441 18262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:44883
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.126.keytab
--principal=oryx/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:44883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:25.553680 18262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:25.554960 18262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:25.566615 18268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:25.566669 18267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:25.567221 18270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:25.567543 18262 server_base.cc:1034] running on GCE node
I20260925 09:27:25.568372 18262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:25.569732 18262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:25.570978 18262 hybrid_clock.cc:648] HybridClock initialized: now 1790328445570919 us; error 43 us; skew 500 ppm
I20260925 09:27:25.571281 18262 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:25 dist-test-slave-9pj8 krb5kdc[18249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328445, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:25.580202 18262 init.cc:377] Logged in from keytab as oryx/127.10.114.126@KRBTEST.COM (short username oryx)
I20260925 09:27:25.581501 18262 webserver.cc:460] Webserver started at http://127.10.114.126:39277/ using document root <none> and password file <none>
I20260925 09:27:25.582136 18262 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:25.582253 18262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:25.582542 18262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:25.585279 18262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "de25c483907244babf8cbcabfaeca842"
format_stamp: "Formatted at 2026-09-25 09:27:25 on dist-test-slave-9pj8"
server_key: "686405b166189fd0d0dc5b9d30e2e480"
server_key_iv: "92bd19c49a15e313ddc96f3aed336d49"
server_key_version: "encryptionkey@0"
I20260925 09:27:25.586046 18262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "de25c483907244babf8cbcabfaeca842"
format_stamp: "Formatted at 2026-09-25 09:27:25 on dist-test-slave-9pj8"
server_key: "686405b166189fd0d0dc5b9d30e2e480"
server_key_iv: "92bd19c49a15e313ddc96f3aed336d49"
server_key_version: "encryptionkey@0"
I20260925 09:27:25.590595 18262 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260925 09:27:25.593760 18277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:25.594923 18262 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260925 09:27:25.595088 18262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "de25c483907244babf8cbcabfaeca842"
format_stamp: "Formatted at 2026-09-25 09:27:25 on dist-test-slave-9pj8"
server_key: "686405b166189fd0d0dc5b9d30e2e480"
server_key_iv: "92bd19c49a15e313ddc96f3aed336d49"
server_key_version: "encryptionkey@0"
I20260925 09:27:25.595288 18262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:25.605616 18262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:25.609365 18262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:25.609601 18262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:25.630093 18262 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:44883
I20260925 09:27:25.630116 18338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:44883 every 8 connection(s)
I20260925 09:27:25.631310 18262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:27:25.634816 18339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:25.640036 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 18262
I20260925 09:27:25.640223 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:27:25.641016 10697 external_mini_cluster.cc:1428] Setting key 424e2f9b4c32b5fafaf671b71ac8ceaa
I20260925 09:27:25.645542 18339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842: Bootstrap starting.
I20260925 09:27:25.648443 18339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:25.649369 18339 log.cc:824] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:25.651805 18339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842: No bootstrap required, opened a new log
Sep 25 09:27:25 dist-test-slave-9pj8 krb5kdc[18249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM
I20260925 09:27:25.660382 18339 raft_consensus.cc:348] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de25c483907244babf8cbcabfaeca842" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44883 } }
I20260925 09:27:25.660836 18339 raft_consensus.cc:374] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:25.660941 18339 raft_consensus.cc:729] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de25c483907244babf8cbcabfaeca842, State: Initialized, Role: FOLLOWER
I20260925 09:27:25.661477 18339 consensus_queue.cc:260] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de25c483907244babf8cbcabfaeca842" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44883 } }
I20260925 09:27:25.661726 18339 raft_consensus.cc:388] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:27:25.661880 18339 raft_consensus.cc:482] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:27:25.662045 18339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:25.662338 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:25.643118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59654 (local address 127.10.114.126:44883)
0925 09:27:25.643938 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:25.643949 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:25.643986 (+ 37us) server_negotiation.cc:407] Connection header received
0925 09:27:25.645491 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:25.645511 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:25.645910 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:25.646261 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:25.647628 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:25.649003 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:25.650549 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:25.651142 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:25.651310 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:25.655379 (+ 4069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:25.655386 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:25.655420 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:25.655450 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:25.658247 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:25.658975 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:25.658981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:25.658985 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:25.659130 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:25.659494 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:25.659497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:25.659499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:25.659985 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:25.660170 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:25.660705 (+ 535us) server_negotiation.cc:299] Negotiation successful
0925 09:27:25.661161 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":349,"threads_started":1}
I20260925 09:27:25.664260 18339 raft_consensus.cc:504] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de25c483907244babf8cbcabfaeca842" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44883 } }
I20260925 09:27:25.664745 18339 leader_election.cc:302] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de25c483907244babf8cbcabfaeca842; no voters:
I20260925 09:27:25.665266 18339 leader_election.cc:288] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:27:25.665376 18344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:25.666040 18344 raft_consensus.cc:686] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842 [term 1 LEADER]: Becoming Leader. State: Replica: de25c483907244babf8cbcabfaeca842, State: Running, Role: LEADER
I20260925 09:27:25.666484 18344 consensus_queue.cc:237] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de25c483907244babf8cbcabfaeca842" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44883 } }
I20260925 09:27:25.667140 18339 sys_catalog.cc:564] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:27:25.669104 18345 sys_catalog.cc:455] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de25c483907244babf8cbcabfaeca842" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de25c483907244babf8cbcabfaeca842" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44883 } } }
I20260925 09:27:25.669450 18345 sys_catalog.cc:458] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:25.669907 18346 sys_catalog.cc:455] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de25c483907244babf8cbcabfaeca842. Latest consensus state: current_term: 1 leader_uuid: "de25c483907244babf8cbcabfaeca842" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de25c483907244babf8cbcabfaeca842" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44883 } } }
I20260925 09:27:25.670184 18346 sys_catalog.cc:458] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:25.670506 18350 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:27:25.676864 18350 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:27:25.685230 18350 catalog_manager.cc:1347] Generated new cluster ID: a42bf45285104db9bb592df38951f429
I20260925 09:27:25.685346 18350 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:27:25.698899 18350 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:27:25.700347 18350 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:27:25.709610 18350 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842: Generated new TSK 0
I20260925 09:27:25.710377 18350 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.65.keytab.
Entry for principal oryx/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.65.keytab.
I20260925 09:27:25.765415 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:44883
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.65.keytab
--principal=oryx/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:26.076824 18365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:26.077095 18365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:26.077250 18365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:26.084174 18365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:26.084293 18365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:26.084434 18365 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 09:27:26.084553 18365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:27:26.094036 18365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.65.keytab
--principal=oryx/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.114.126:44883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:26.095252 18365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:26.096642 18365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:26.108920 18373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:26.109010 18371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:26.108983 18370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:26.109861 18365 server_base.cc:1034] running on GCE node
I20260925 09:27:26.110345 18365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:26.111078 18365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:26.112258 18365 hybrid_clock.cc:648] HybridClock initialized: now 1790328446112193 us; error 44 us; skew 500 ppm
I20260925 09:27:26.112515 18365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:26 dist-test-slave-9pj8 krb5kdc[18249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328446, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:26.115296 18365 init.cc:377] Logged in from keytab as oryx/127.10.114.65@KRBTEST.COM (short username oryx)
I20260925 09:27:26.116650 18365 webserver.cc:460] Webserver started at http://127.10.114.65:39203/ using document root <none> and password file <none>
I20260925 09:27:26.117277 18365 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:26.117393 18365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:26.117688 18365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:26.120433 18365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "18099eba8274427087a5f753f8f86e75"
format_stamp: "Formatted at 2026-09-25 09:27:26 on dist-test-slave-9pj8"
server_key: "1da621f32d3e880a6796f84bd2677636"
server_key_iv: "ef1029cb91b92870298750600d48823b"
server_key_version: "encryptionkey@0"
I20260925 09:27:26.121176 18365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "18099eba8274427087a5f753f8f86e75"
format_stamp: "Formatted at 2026-09-25 09:27:26 on dist-test-slave-9pj8"
server_key: "1da621f32d3e880a6796f84bd2677636"
server_key_iv: "ef1029cb91b92870298750600d48823b"
server_key_version: "encryptionkey@0"
I20260925 09:27:26.125905 18365 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260925 09:27:26.129050 18380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:26.130321 18365 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 09:27:26.130504 18365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "18099eba8274427087a5f753f8f86e75"
format_stamp: "Formatted at 2026-09-25 09:27:26 on dist-test-slave-9pj8"
server_key: "1da621f32d3e880a6796f84bd2677636"
server_key_iv: "ef1029cb91b92870298750600d48823b"
server_key_version: "encryptionkey@0"
I20260925 09:27:26.130730 18365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:26.145597 18365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:26.149319 18365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:26.149597 18365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:26.152035 18365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:26.152181 18365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:26.152343 18365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:26.152462 18365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Sep 25 09:27:26 dist-test-slave-9pj8 krb5kdc[18249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328446, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM
I20260925 09:27:26.175280 18389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:26.153033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:44883 (local address 127.10.114.65:33277)
0925 09:27:26.153801 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:26.153848 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:26.155887 (+ 2039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:26.156601 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:26.156611 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:26.157318 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:26.158050 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:26.158093 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:26.159722 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:26.159733 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:26.161841 (+ 2108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:26.161852 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:26.162883 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:26.162893 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:26.163219 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:26.163894 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:26.163928 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:26.166979 (+ 3051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:26.170508 (+ 3529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:26.170518 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:26.170531 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:26.170958 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:26.171451 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:26.171471 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:26.171476 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:26.171724 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:26.172447 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:26.172452 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:26.172941 (+ 489us) client_negotiation.cc:769] Sending connection context
0925 09:27:26.173292 (+ 351us) client_negotiation.cc:240] Negotiation successful
0925 09:27:26.173782 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":329,"threads_started":1}
I20260925 09:27:26.176303 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:26.152537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:33277 (local address 127.10.114.126:44883)
0925 09:27:26.152712 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:26.152718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:26.153957 (+ 1239us) server_negotiation.cc:407] Connection header received
0925 09:27:26.156148 (+ 2191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:26.156156 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:26.156257 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:26.156374 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:26.158265 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:26.159555 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:26.161972 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:26.162648 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:26.162789 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:26.167189 (+ 4400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:26.167194 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:26.167197 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:26.167228 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:26.170397 (+ 3169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:26.171146 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:26.171153 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:26.171159 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:26.171333 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:26.171864 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:26.171869 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:26.171872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:26.172233 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:26.172373 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:26.175821 (+ 3448us) server_negotiation.cc:299] Negotiation successful
0925 09:27:26.176055 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 09:27:26.192487 18365 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:39449
I20260925 09:27:26.192569 18499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:39449 every 8 connection(s)
I20260925 09:27:26.193786 18365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:27:26.194672 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 18365
I20260925 09:27:26.194893 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:27:26.195871 10697 external_mini_cluster.cc:1428] Setting key 378c0bd90714a2204dbcd261f84d5c1c
I20260925 09:27:26.195899 18500 heartbeater.cc:347] Connected to a master server at 127.10.114.126:44883
I20260925 09:27:26.196139 18500 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:26.196980 18500 heartbeater.cc:511] Master 127.10.114.126:44883 requested a full tablet report, sending...
I20260925 09:27:26.197173 18473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:26.182974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:44883 (local address 127.10.114.65:36367)
0925 09:27:26.184110 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:26.184127 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:26.184274 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:26.184554 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:26.184560 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:26.184963 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:26.185321 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:26.185338 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:26.186591 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:26.186601 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:26.188128 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:26.188168 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:26.189193 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:26.189197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:26.189397 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:26.190167 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:26.190194 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:26.191139 (+ 945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:26.194175 (+ 3036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:26.194180 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:26.194184 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:26.194732 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:26.195132 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:26.195136 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:26.195140 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:26.195273 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:26.195959 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:26.195963 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:26.196159 (+ 196us) client_negotiation.cc:769] Sending connection context
0925 09:27:26.196337 (+ 178us) client_negotiation.cc:240] Negotiation successful
0925 09:27:26.196669 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":993,"thread_start_us":787,"threads_started":1}
I20260925 09:27:26.197973 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:26.182501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:36367 (local address 127.10.114.126:44883)
0925 09:27:26.182720 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:26.182726 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:26.184211 (+ 1485us) server_negotiation.cc:407] Connection header received
0925 09:27:26.184380 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:26.184386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:26.184445 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:26.184526 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:26.185526 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:26.186389 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:26.188341 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:26.188844 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:26.188968 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:26.191294 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:26.191300 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:26.191305 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:26.191339 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:26.194030 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:26.194883 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:26.194890 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:26.194896 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:26.195029 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:26.195392 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:26.195397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:26.195400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:26.195733 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:26.195901 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:26.197431 (+ 1530us) server_negotiation.cc:299] Negotiation successful
0925 09:27:26.197653 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 09:27:26.199509 18294 ts_manager.cc:194] Registered new tserver with Master: 18099eba8274427087a5f753f8f86e75 (127.10.114.65:39449)
I20260925 09:27:26.201483 18294 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:33277
WARNING: no policy specified for oryx/127.10.114.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.114.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.66.keytab.
Entry for principal oryx/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.66.keytab.
I20260925 09:27:26.246865 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:44883
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.66.keytab
--principal=oryx/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:26.561486 18504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:26.561794 18504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:26.561961 18504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:26.569195 18504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:26.569308 18504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:26.569447 18504 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 09:27:26.569571 18504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:27:26.579036 18504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.66.keytab
--principal=oryx/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.114.126:44883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:26.580259 18504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:26.581488 18504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:26.593864 18512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:26.593938 18510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:26.593912 18509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:26.595026 18504 server_base.cc:1034] running on GCE node
I20260925 09:27:26.595647 18504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:26.596604 18504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:26.597851 18504 hybrid_clock.cc:648] HybridClock initialized: now 1790328446597784 us; error 42 us; skew 500 ppm
I20260925 09:27:26.598138 18504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:26 dist-test-slave-9pj8 krb5kdc[18249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328446, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:26.601166 18504 init.cc:377] Logged in from keytab as oryx/127.10.114.66@KRBTEST.COM (short username oryx)
I20260925 09:27:26.602553 18504 webserver.cc:460] Webserver started at http://127.10.114.66:37647/ using document root <none> and password file <none>
I20260925 09:27:26.603176 18504 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:26.603256 18504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:26.603504 18504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:26.606233 18504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "4ef88ee04cb149f7982c63065d327f9d"
format_stamp: "Formatted at 2026-09-25 09:27:26 on dist-test-slave-9pj8"
server_key: "637a15310c1d6b160c142289d2b2eceb"
server_key_iv: "0f7d23b1b9d4cd19a3961d083fca96b5"
server_key_version: "encryptionkey@0"
I20260925 09:27:26.606941 18504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "4ef88ee04cb149f7982c63065d327f9d"
format_stamp: "Formatted at 2026-09-25 09:27:26 on dist-test-slave-9pj8"
server_key: "637a15310c1d6b160c142289d2b2eceb"
server_key_iv: "0f7d23b1b9d4cd19a3961d083fca96b5"
server_key_version: "encryptionkey@0"
I20260925 09:27:26.611600 18504 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 09:27:26.614885 18519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:26.616196 18504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:27:26.616318 18504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "4ef88ee04cb149f7982c63065d327f9d"
format_stamp: "Formatted at 2026-09-25 09:27:26 on dist-test-slave-9pj8"
server_key: "637a15310c1d6b160c142289d2b2eceb"
server_key_iv: "0f7d23b1b9d4cd19a3961d083fca96b5"
server_key_version: "encryptionkey@0"
I20260925 09:27:26.616487 18504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:26.648348 18504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:26.652594 18504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:26.652808 18504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:26.655102 18504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:26.655203 18504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:26.655334 18504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:26.655402 18504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 09:27:26 dist-test-slave-9pj8 krb5kdc[18249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328446, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM
I20260925 09:27:26.677745 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:26.656771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:58659 (local address 127.10.114.126:44883)
0925 09:27:26.656935 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:26.656941 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:26.657341 (+ 400us) server_negotiation.cc:407] Connection header received
0925 09:27:26.660390 (+ 3049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:26.660396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:26.660480 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:26.660595 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:26.662642 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:26.664014 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:26.666198 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:26.667035 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:26.667193 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:26.670616 (+ 3423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:26.670624 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:26.670632 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:26.670681 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:26.674053 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:26.675135 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:26.675143 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:26.675146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:26.675274 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:26.675655 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:26.675661 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:26.675664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:26.676075 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:26.676219 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:26.677135 (+ 916us) server_negotiation.cc:299] Negotiation successful
0925 09:27:26.677379 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 09:27:26.678669 18528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:26.656325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:44883 (local address 127.10.114.66:58659)
0925 09:27:26.657171 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:26.657224 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:26.660047 (+ 2823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:26.660869 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:26.660885 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:26.661645 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:26.662451 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:26.662469 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:26.664193 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:26.664204 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:26.666005 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:26.666024 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:26.667321 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:26.667330 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:26.667511 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:26.668215 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:26.668234 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:26.670382 (+ 2148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:26.674165 (+ 3783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:26.674172 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:26.674186 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:26.674999 (+ 813us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:26.675378 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:26.675384 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:26.675388 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:26.675552 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:26.676290 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:26.676295 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:26.676740 (+ 445us) client_negotiation.cc:769] Sending connection context
0925 09:27:26.677063 (+ 323us) client_negotiation.cc:240] Negotiation successful
0925 09:27:26.677554 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":312,"threads_started":1}
I20260925 09:27:26.706194 18578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:26.683929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:44883 (local address 127.10.114.66:55731)
0925 09:27:26.692387 (+ 8458us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:26.692407 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:26.692631 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:26.693154 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:26.693161 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:26.693852 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:26.694128 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:26.694136 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:26.695105 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:26.695110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:26.696277 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:26.696295 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:26.697284 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:26.697289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:26.697441 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:26.698069 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:26.698086 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:26.698823 (+ 737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:26.703332 (+ 4509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:26.703336 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:26.703338 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:26.703753 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:26.704248 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:26.704255 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:26.704258 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:26.704388 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:26.705220 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:26.705223 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:26.705412 (+ 189us) client_negotiation.cc:769] Sending connection context
0925 09:27:26.705605 (+ 193us) client_negotiation.cc:240] Negotiation successful
0925 09:27:26.705904 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8304,"thread_start_us":8086,"threads_started":1}
I20260925 09:27:26.706532 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:26.683820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:55731 (local address 127.10.114.126:44883)
0925 09:27:26.684011 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:26.684018 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:26.692510 (+ 8492us) server_negotiation.cc:407] Connection header received
0925 09:27:26.692796 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:26.692810 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:26.692874 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:26.692927 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:26.694256 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:26.694970 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:26.696434 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:26.697101 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:26.697242 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:26.699032 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:26.699042 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:26.699049 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:26.699095 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:26.703199 (+ 4104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:26.703944 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:26.703951 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:26.703957 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:26.704134 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:26.704563 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:26.704571 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:26.704576 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:26.705003 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:26.705151 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:26.705751 (+ 600us) server_negotiation.cc:299] Negotiation successful
0925 09:27:26.706065 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 09:27:26.712175 18504 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:46661
I20260925 09:27:26.712203 18638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:46661 every 8 connection(s)
I20260925 09:27:26.713299 18504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:27:26.714260 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 18504
I20260925 09:27:26.714423 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:27:26.715026 10697 external_mini_cluster.cc:1428] Setting key 49503f1b2637413c263e08a3f898c6c1
I20260925 09:27:26.715195 18639 heartbeater.cc:347] Connected to a master server at 127.10.114.126:44883
I20260925 09:27:26.715368 18639 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:26.716029 18639 heartbeater.cc:511] Master 127.10.114.126:44883 requested a full tablet report, sending...
I20260925 09:27:26.717267 18294 ts_manager.cc:194] Registered new tserver with Master: 4ef88ee04cb149f7982c63065d327f9d (127.10.114.66:46661)
I20260925 09:27:26.718005 18294 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:58659
WARNING: no policy specified for oryx/127.10.114.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.114.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.67.keytab.
Entry for principal oryx/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.67.keytab.
I20260925 09:27:26.758383 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:44883
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.67.keytab
--principal=oryx/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:27.068461 18643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:27.068737 18643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:27.068892 18643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:27.075991 18643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:27.076138 18643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:27.076261 18643 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 09:27:27.076380 18643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:27:27.085681 18643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.67.keytab
--principal=oryx/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.114.126:44883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:27.086853 18643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:27.088124 18643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:27.100209 18649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:27.100266 18648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:27.100291 18651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:27.101058 18643 server_base.cc:1034] running on GCE node
I20260925 09:27:27.101653 18643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:27.102501 18643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:27.103729 18643 hybrid_clock.cc:648] HybridClock initialized: now 1790328447103676 us; error 52 us; skew 500 ppm
I20260925 09:27:27.104025 18643 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:27 dist-test-slave-9pj8 krb5kdc[18249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328447, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:27.107033 18643 init.cc:377] Logged in from keytab as oryx/127.10.114.67@KRBTEST.COM (short username oryx)
I20260925 09:27:27.108549 18643 webserver.cc:460] Webserver started at http://127.10.114.67:43885/ using document root <none> and password file <none>
I20260925 09:27:27.109172 18643 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:27.109301 18643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:27.109596 18643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:27.112329 18643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "b0627b57c8454d46841f7d512624150a"
format_stamp: "Formatted at 2026-09-25 09:27:27 on dist-test-slave-9pj8"
server_key: "7134baaeb52ac1be196bbbcd5739dcf2"
server_key_iv: "e55acf23c491031a140c6540191e2df7"
server_key_version: "encryptionkey@0"
I20260925 09:27:27.113025 18643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "b0627b57c8454d46841f7d512624150a"
format_stamp: "Formatted at 2026-09-25 09:27:27 on dist-test-slave-9pj8"
server_key: "7134baaeb52ac1be196bbbcd5739dcf2"
server_key_iv: "e55acf23c491031a140c6540191e2df7"
server_key_version: "encryptionkey@0"
I20260925 09:27:27.117616 18643 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:27:27.120795 18658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:27.121994 18643 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:27:27.122151 18643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "b0627b57c8454d46841f7d512624150a"
format_stamp: "Formatted at 2026-09-25 09:27:27 on dist-test-slave-9pj8"
server_key: "7134baaeb52ac1be196bbbcd5739dcf2"
server_key_iv: "e55acf23c491031a140c6540191e2df7"
server_key_version: "encryptionkey@0"
I20260925 09:27:27.122375 18643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:27.149576 18643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:27.153189 18643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:27.153434 18643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:27.155956 18643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:27.156050 18643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:27.156188 18643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:27.156268 18643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 09:27:27 dist-test-slave-9pj8 krb5kdc[18249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328447, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM
I20260925 09:27:27.182111 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.156479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:53351 (local address 127.10.114.126:44883)
0925 09:27:27.156662 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:27.156669 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:27.158037 (+ 1368us) server_negotiation.cc:407] Connection header received
0925 09:27:27.160012 (+ 1975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:27.160018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:27.160105 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:27.160239 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:27.162535 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.163901 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.167370 (+ 3469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.168052 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.168202 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.173923 (+ 5721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:27.173932 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:27.173938 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:27.173981 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:27.177572 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:27.178720 (+ 1148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:27.178728 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:27.178733 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:27.178926 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:27.179556 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:27.179562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:27.179567 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:27.179959 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:27.180144 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:27.181196 (+ 1052us) server_negotiation.cc:299] Negotiation successful
0925 09:27:27.181541 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:27:27.182761 18667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.156995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:44883 (local address 127.10.114.67:53351)
0925 09:27:27.157861 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:27.157909 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:27.159702 (+ 1793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:27.160544 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:27.160556 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:27.161367 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:27.162330 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:27.162360 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.164083 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.164092 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:27.167128 (+ 3036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:27.167146 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.168390 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.168403 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:27.168846 (+ 443us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.169693 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:27.169730 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:27.173583 (+ 3853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:27.177733 (+ 4150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:27.177747 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:27.177772 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:27.178526 (+ 754us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:27.179069 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:27.179079 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:27.179085 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:27.179406 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:27.180137 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:27.180147 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:27.180694 (+ 547us) client_negotiation.cc:769] Sending connection context
0925 09:27:27.181063 (+ 369us) client_negotiation.cc:240] Negotiation successful
0925 09:27:27.181609 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":388,"threads_started":1}
I20260925 09:27:27.204147 18500 heartbeater.cc:503] Master 127.10.114.126:44883 was elected leader, sending a full tablet report...
I20260925 09:27:27.208304 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.188024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:40341 (local address 127.10.114.126:44883)
0925 09:27:27.188198 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:27.188205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:27.192381 (+ 4176us) server_negotiation.cc:407] Connection header received
0925 09:27:27.192564 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:27.192573 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:27.192686 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:27.192787 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:27.194370 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.195585 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.198193 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.198860 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.198980 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.201752 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:27.201759 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:27.201763 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:27.201801 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:27.205097 (+ 3296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:27.206141 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:27.206148 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:27.206152 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:27.206376 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:27.206854 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:27.206859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:27.206862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:27.207104 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:27.207253 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:27.207691 (+ 438us) server_negotiation.cc:299] Negotiation successful
0925 09:27:27.207930 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 09:27:27.208468 18743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.188207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:44883 (local address 127.10.114.67:40341)
0925 09:27:27.192261 (+ 4054us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:27.192282 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:27.192416 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:27.192997 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:27.193004 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:27.193687 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:27.194162 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:27.194176 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.195864 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.195870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:27.198001 (+ 2131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:27.198025 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.199107 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.199118 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:27.199395 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.200323 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:27.200357 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:27.201583 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:27.205350 (+ 3767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:27.205359 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:27.205363 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:27.205989 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:27.206532 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:27.206539 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:27.206545 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:27.206732 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:27.207231 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:27.207236 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:27.207422 (+ 186us) client_negotiation.cc:769] Sending connection context
0925 09:27:27.207566 (+ 144us) client_negotiation.cc:240] Negotiation successful
0925 09:27:27.207937 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3875,"thread_start_us":3723,"threads_started":1}
I20260925 09:27:27.209054 18643 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:46043
I20260925 09:27:27.209075 18777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:46043 every 8 connection(s)
I20260925 09:27:27.210489 18643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:27:27.213126 18778 heartbeater.cc:347] Connected to a master server at 127.10.114.126:44883
I20260925 09:27:27.213315 18778 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:27.213904 18778 heartbeater.cc:511] Master 127.10.114.126:44883 requested a full tablet report, sending...
I20260925 09:27:27.215049 18294 ts_manager.cc:194] Registered new tserver with Master: b0627b57c8454d46841f7d512624150a (127.10.114.67:46043)
I20260925 09:27:27.216164 18294 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:53351
I20260925 09:27:27.216709 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 18643
I20260925 09:27:27.216871 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:27:27.217343 10697 external_mini_cluster.cc:1428] Setting key 5b1e90849f00eb94334191e77d13f6d8
I20260925 09:27:27.220220 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 09:27:27 dist-test-slave-9pj8 krb5kdc[18249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328447, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 09:27:27 dist-test-slave-9pj8 krb5kdc[18249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:27 dist-test-slave-9pj8 krb5kdc[18249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 09:27:27.266378 18342 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:27.256432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59670 (local address 127.10.114.126:44883)
0925 09:27:27.256661 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:27.256667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:27.256856 (+ 189us) server_negotiation.cc:407] Connection header received
0925 09:27:27.257048 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:27.257053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:27.257122 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:27.257227 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:27.258495 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.259236 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.260521 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.260925 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.261053 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.266146 (+ 5093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59670: BlockingRecv error: recv got EOF from 127.0.0.1:59670 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 25 09:27:27 dist-test-slave-9pj8 krb5kdc[18249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328447, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM
I20260925 09:27:27.281937 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.269554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59676 (local address 127.10.114.126:44883)
0925 09:27:27.269722 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:27.269728 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:27.269857 (+ 129us) server_negotiation.cc:407] Connection header received
0925 09:27:27.269978 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:27.269981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:27.270060 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:27.270171 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:27.271272 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.271984 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.273289 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.273684 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.273778 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.277164 (+ 3386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:27.277168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:27.277171 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:27.277199 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:27.279760 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:27.280356 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:27.280359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:27.280361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:27.280461 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:27.280839 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:27.280842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:27.280844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:27.281102 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:27.281284 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:27.281470 (+ 186us) server_negotiation.cc:299] Negotiation successful
0925 09:27:27.281606 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:27:27.285647 18294 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:27:27.292183 18294 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:27:27.322602 18800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.304009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38208 (local address 127.10.114.66:46661)
0925 09:27:27.312260 (+ 8251us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:27.312271 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:27.315216 (+ 2945us) server_negotiation.cc:407] Connection header received
0925 09:27:27.315313 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:27.315320 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:27.315528 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:27.315720 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:27.316410 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.317378 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.319935 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.321216 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.321343 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.321533 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:27.322104 (+ 571us) server_negotiation.cc:299] Negotiation successful
0925 09:27:27.322298 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8033,"thread_start_us":7899,"threads_started":1}
I20260925 09:27:27.322674 18798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.303663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:46661 (local address 127.0.0.1:38208)
0925 09:27:27.315045 (+ 11382us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:27.315061 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:27.315158 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:27.315834 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:27.315838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:27.315858 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:27.316255 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:27.316266 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.317536 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.317541 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:27.319760 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:27.319812 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.321669 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.321675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:27.321847 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.321902 (+ 55us) client_negotiation.cc:769] Sending connection context
0925 09:27:27.322027 (+ 125us) client_negotiation.cc:240] Negotiation successful
0925 09:27:27.322191 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11284,"mutex_wait_us":36,"thread_start_us":11169,"threads_started":1}
I20260925 09:27:27.322901 18797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.303358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:39449 (local address 127.0.0.1:52496)
0925 09:27:27.314146 (+ 10788us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:27.314166 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:27.314281 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:27.315576 (+ 1295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:27.315580 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:27.315604 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:27.316094 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:27.316106 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.318139 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.318144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:27.320372 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:27.320382 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.322049 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.322055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:27.322212 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.322245 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 09:27:27.322322 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 09:27:27.322437 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10646,"mutex_wait_us":50,"thread_start_us":10921,"threads_started":1}
I20260925 09:27:27.323503 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.303749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52496 (local address 127.10.114.65:39449)
0925 09:27:27.314844 (+ 11095us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:27.314852 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:27.314876 (+ 24us) server_negotiation.cc:407] Connection header received
0925 09:27:27.314958 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:27.314965 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:27.315374 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:27.315585 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:27.316292 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.318006 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.320500 (+ 2494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.321847 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.321983 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.322188 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:27.323195 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0925 09:27:27.323312 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10918,"thread_start_us":10745,"threads_started":1}
I20260925 09:27:27.326449 18569 tablet_service.cc:1467] Processing CreateTablet for tablet 1d2034121a6b47e185712c8bef86655a (DEFAULT_TABLE table=test-table [id=cb377469bb5947d193e667b6d98feed7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:27.326498 18430 tablet_service.cc:1467] Processing CreateTablet for tablet 1d2034121a6b47e185712c8bef86655a (DEFAULT_TABLE table=test-table [id=cb377469bb5947d193e667b6d98feed7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:27.327847 18430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d2034121a6b47e185712c8bef86655a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:27.327859 18569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d2034121a6b47e185712c8bef86655a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:27.329213 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.303306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:46043 (local address 127.0.0.1:49322)
0925 09:27:27.313225 (+ 9919us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:27.313289 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:27.313517 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:27.315325 (+ 1808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:27.315331 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:27.315357 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:27.315726 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:27.315744 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.317747 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.317752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:27.320368 (+ 2616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:27.320382 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.328396 (+ 8014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.328402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:27.328563 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.328597 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 09:27:27.328692 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 09:27:27.328813 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9613,"mutex_wait_us":139,"thread_start_us":9462,"threads_started":1}
I20260925 09:27:27.329243 18801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.304598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49322 (local address 127.10.114.67:46043)
0925 09:27:27.312804 (+ 8206us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:27.312811 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:27.313401 (+ 590us) server_negotiation.cc:407] Connection header received
0925 09:27:27.313778 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:27.313789 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:27.314103 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:27.315089 (+ 986us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:27.315970 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.317576 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.322272 (+ 4696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.328194 (+ 5922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.328342 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.328555 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:27.328758 (+ 203us) server_negotiation.cc:299] Negotiation successful
0925 09:27:27.328936 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8029,"thread_start_us":7817,"threads_started":1}
I20260925 09:27:27.332273 18708 tablet_service.cc:1467] Processing CreateTablet for tablet 1d2034121a6b47e185712c8bef86655a (DEFAULT_TABLE table=test-table [id=cb377469bb5947d193e667b6d98feed7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:27.333331 18708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d2034121a6b47e185712c8bef86655a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:27.341910 18802 tablet_bootstrap.cc:492] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d: Bootstrap starting.
I20260925 09:27:27.343487 18803 tablet_bootstrap.cc:492] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75: Bootstrap starting.
I20260925 09:27:27.344254 18804 tablet_bootstrap.cc:492] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a: Bootstrap starting.
I20260925 09:27:27.346560 18803 tablet_bootstrap.cc:654] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:27.347146 18802 tablet_bootstrap.cc:654] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:27.347325 18804 tablet_bootstrap.cc:654] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:27.347625 18803 log.cc:824] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:27.348389 18804 log.cc:824] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:27.348385 18802 log.cc:824] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:27.350268 18803 tablet_bootstrap.cc:492] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75: No bootstrap required, opened a new log
I20260925 09:27:27.350515 18803 ts_tablet_manager.cc:1397] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260925 09:27:27.351514 18802 tablet_bootstrap.cc:492] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d: No bootstrap required, opened a new log
I20260925 09:27:27.351852 18802 ts_tablet_manager.cc:1397] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.002s
I20260925 09:27:27.358016 18803 raft_consensus.cc:348] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } }
I20260925 09:27:27.358213 18804 tablet_bootstrap.cc:492] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a: No bootstrap required, opened a new log
I20260925 09:27:27.358369 18803 raft_consensus.cc:374] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:27.358464 18804 ts_tablet_manager.cc:1397] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.003s
I20260925 09:27:27.358487 18803 raft_consensus.cc:729] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18099eba8274427087a5f753f8f86e75, State: Initialized, Role: FOLLOWER
I20260925 09:27:27.359098 18803 consensus_queue.cc:260] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } }
I20260925 09:27:27.360721 18803 ts_tablet_manager.cc:1428] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260925 09:27:27.363127 18802 raft_consensus.cc:348] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } }
I20260925 09:27:27.363492 18802 raft_consensus.cc:374] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:27.363579 18802 raft_consensus.cc:729] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ef88ee04cb149f7982c63065d327f9d, State: Initialized, Role: FOLLOWER
I20260925 09:27:27.364254 18802 consensus_queue.cc:260] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } }
I20260925 09:27:27.365849 18802 ts_tablet_manager.cc:1428] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260925 09:27:27.366092 18639 heartbeater.cc:503] Master 127.10.114.126:44883 was elected leader, sending a full tablet report...
I20260925 09:27:27.368374 18804 raft_consensus.cc:348] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } }
I20260925 09:27:27.368755 18804 raft_consensus.cc:374] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:27.368842 18804 raft_consensus.cc:729] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0627b57c8454d46841f7d512624150a, State: Initialized, Role: FOLLOWER
I20260925 09:27:27.369383 18804 consensus_queue.cc:260] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } }
I20260925 09:27:27.370358 18778 heartbeater.cc:503] Master 127.10.114.126:44883 was elected leader, sending a full tablet report...
I20260925 09:27:27.370893 18804 ts_tablet_manager.cc:1428] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a: Time spent starting tablet: real 0.012s user 0.013s sys 0.001s
I20260925 09:27:27.405519 18809 raft_consensus.cc:482] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:27:27.405813 18809 raft_consensus.cc:504] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } }
I20260925 09:27:27.407437 18809 leader_election.cc:288] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0627b57c8454d46841f7d512624150a (127.10.114.67:46043), 18099eba8274427087a5f753f8f86e75 (127.10.114.65:39449)
I20260925 09:27:27.413954 18801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.407741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:44133 (local address 127.10.114.67:46043)
0925 09:27:27.408006 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:27.408015 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:27.408706 (+ 691us) server_negotiation.cc:407] Connection header received
0925 09:27:27.408846 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:27.408850 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:27.408919 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:27.409012 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:27.409784 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.410593 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.412397 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.413088 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.413278 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.413349 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:27.413596 (+ 247us) server_negotiation.cc:299] Negotiation successful
0925 09:27:27.413710 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260925 09:27:27.413954 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.408159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:45895 (local address 127.10.114.65:39449)
0925 09:27:27.408330 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:27.408335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:27.408579 (+ 244us) server_negotiation.cc:407] Connection header received
0925 09:27:27.408838 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:27.408843 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:27.408919 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:27.409012 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:27.409728 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.410521 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.412319 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.413060 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.413253 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.413328 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:27.413588 (+ 260us) server_negotiation.cc:299] Negotiation successful
0925 09:27:27.413715 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 09:27:27.414131 18812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.407869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:46043 (local address 127.10.114.66:44133)
0925 09:27:27.408609 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:27.408633 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:27.408737 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:27.409053 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:27.409058 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:27.409089 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:27.409597 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:27.409605 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.410701 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.410704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:27.412261 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:27.412272 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.413308 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.413314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:27.413459 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.413479 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 09:27:27.413574 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 09:27:27.413696 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"mutex_wait_us":24,"spinlock_wait_cycles":19328,"thread_start_us":472,"threads_started":1}
I20260925 09:27:27.414338 18811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.407744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:39449 (local address 127.10.114.66:45895)
0925 09:27:27.408481 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:27.408500 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:27.408658 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:27.409062 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:27.409065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:27.409125 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:27.409571 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:27.409588 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.410656 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.410664 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:27.412164 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:27.412176 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.413282 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.413288 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:27.413444 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.413468 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 09:27:27.413565 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 09:27:27.413683 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":5,"spinlock_wait_cycles":38016,"thread_start_us":304,"threads_started":1}
I20260925 09:27:27.415068 18728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d2034121a6b47e185712c8bef86655a" candidate_uuid: "4ef88ee04cb149f7982c63065d327f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0627b57c8454d46841f7d512624150a" is_pre_election: true
I20260925 09:27:27.415112 18450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d2034121a6b47e185712c8bef86655a" candidate_uuid: "4ef88ee04cb149f7982c63065d327f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18099eba8274427087a5f753f8f86e75" is_pre_election: true
I20260925 09:27:27.415632 18728 raft_consensus.cc:2454] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ef88ee04cb149f7982c63065d327f9d in term 0.
I20260925 09:27:27.415628 18450 raft_consensus.cc:2454] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ef88ee04cb149f7982c63065d327f9d in term 0.
I20260925 09:27:27.416260 18523 leader_election.cc:302] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18099eba8274427087a5f753f8f86e75, 4ef88ee04cb149f7982c63065d327f9d; no voters:
I20260925 09:27:27.416589 18809 raft_consensus.cc:2781] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:27:27.416751 18809 raft_consensus.cc:482] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:27:27.416870 18809 raft_consensus.cc:3037] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:27.418901 18809 raft_consensus.cc:504] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } }
I20260925 09:27:27.419423 18809 leader_election.cc:288] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [CANDIDATE]: Term 1 election: Requested vote from peers b0627b57c8454d46841f7d512624150a (127.10.114.67:46043), 18099eba8274427087a5f753f8f86e75 (127.10.114.65:39449)
I20260925 09:27:27.419904 18728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d2034121a6b47e185712c8bef86655a" candidate_uuid: "4ef88ee04cb149f7982c63065d327f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0627b57c8454d46841f7d512624150a"
I20260925 09:27:27.419921 18450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d2034121a6b47e185712c8bef86655a" candidate_uuid: "4ef88ee04cb149f7982c63065d327f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18099eba8274427087a5f753f8f86e75"
I20260925 09:27:27.420099 18728 raft_consensus.cc:3037] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:27.420099 18450 raft_consensus.cc:3037] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:27.423118 18728 raft_consensus.cc:2454] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ef88ee04cb149f7982c63065d327f9d in term 1.
I20260925 09:27:27.423118 18450 raft_consensus.cc:2454] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ef88ee04cb149f7982c63065d327f9d in term 1.
I20260925 09:27:27.423445 18523 leader_election.cc:302] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18099eba8274427087a5f753f8f86e75, 4ef88ee04cb149f7982c63065d327f9d; no voters:
I20260925 09:27:27.423700 18809 raft_consensus.cc:2781] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:27.424116 18809 raft_consensus.cc:686] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [term 1 LEADER]: Becoming Leader. State: Replica: 4ef88ee04cb149f7982c63065d327f9d, State: Running, Role: LEADER
I20260925 09:27:27.424532 18809 consensus_queue.cc:237] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } }
I20260925 09:27:27.428821 18294 catalog_manager.cc:5495] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ef88ee04cb149f7982c63065d327f9d (127.10.114.66). New cstate: current_term: 1 leader_uuid: "4ef88ee04cb149f7982c63065d327f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } health_report { overall_health: HEALTHY } } }
I20260925 09:27:27.444855 18800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.439891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38218 (local address 127.10.114.66:46661)
0925 09:27:27.440050 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:27.440054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:27.440204 (+ 150us) server_negotiation.cc:407] Connection header received
0925 09:27:27.440385 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:27.440389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:27.440458 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:27.440570 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:27:27.440939 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.441631 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.442929 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.443304 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.443391 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.443824 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:27:27.444322 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:27:27.444422 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:27.444598 (+ 176us) server_negotiation.cc:299] Negotiation successful
0925 09:27:27.444660 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 09:27:27.452630 18450 raft_consensus.cc:1261] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75 [term 1 FOLLOWER]: Refusing update from remote peer 4ef88ee04cb149f7982c63065d327f9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:27:27.452629 18728 raft_consensus.cc:1261] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a [term 1 FOLLOWER]: Refusing update from remote peer 4ef88ee04cb149f7982c63065d327f9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:27:27.453621 18809 consensus_queue.cc:1035] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:27:27.453927 18813 consensus_queue.cc:1035] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:27:27.456059 18501 tablet.cc:2369] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:27:27.462627 18779 tablet.cc:2369] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:27:27.464386 18816 mvcc.cc:205] Tried to move back new op lower bound from 7333185320756400128 to 7333185320654491648. Current Snapshot: MvccSnapshot[applied={T|T < 7333185320756400128}]
I20260925 09:27:27.465302 18818 mvcc.cc:205] Tried to move back new op lower bound from 7333185320756400128 to 7333185320654491648. Current Snapshot: MvccSnapshot[applied={T|T < 7333185320756400128}]
I20260925 09:27:27.465323 18817 mvcc.cc:205] Tried to move back new op lower bound from 7333185320756400128 to 7333185320654491648. Current Snapshot: MvccSnapshot[applied={T|T < 7333185320756400128}]
I20260925 09:27:27.478806 18294 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:27:27.479053 18294 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:27:27.482579 18294 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842: Sending DeleteTablet for 3 replicas of tablet 1d2034121a6b47e185712c8bef86655a
I20260925 09:27:27.483654 18430 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d2034121a6b47e185712c8bef86655a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:27 UTC) from {username='oryx', principal='oryx/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:52496
I20260925 09:27:27.483706 18569 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d2034121a6b47e185712c8bef86655a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:27 UTC) from {username='oryx', principal='oryx/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:38208
I20260925 09:27:27.483825 18708 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d2034121a6b47e185712c8bef86655a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:27 UTC) from {username='oryx', principal='oryx/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:49322
I20260925 09:27:27.485477 18294 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:27:27.486352 18294 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:27:27.495666 18429 tablet_service.cc:1467] Processing CreateTablet for tablet 349c4342cf0a44c8aa0dbfc3adef5307 (DEFAULT_TABLE table=test-table [id=324022995e56453a92a378ed9bbda99e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:27.495915 18568 tablet_service.cc:1467] Processing CreateTablet for tablet 349c4342cf0a44c8aa0dbfc3adef5307 (DEFAULT_TABLE table=test-table [id=324022995e56453a92a378ed9bbda99e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:27.496241 18429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 349c4342cf0a44c8aa0dbfc3adef5307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:27.496425 18568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 349c4342cf0a44c8aa0dbfc3adef5307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:27.496330 18707 tablet_service.cc:1467] Processing CreateTablet for tablet 349c4342cf0a44c8aa0dbfc3adef5307 (DEFAULT_TABLE table=test-table [id=324022995e56453a92a378ed9bbda99e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:27.496829 18707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 349c4342cf0a44c8aa0dbfc3adef5307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:27.501067 18803 tablet_bootstrap.cc:492] T 349c4342cf0a44c8aa0dbfc3adef5307 P 18099eba8274427087a5f753f8f86e75: Bootstrap starting.
I20260925 09:27:27.503085 18803 tablet_bootstrap.cc:654] T 349c4342cf0a44c8aa0dbfc3adef5307 P 18099eba8274427087a5f753f8f86e75: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:27.503157 18804 tablet_bootstrap.cc:492] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a: Bootstrap starting.
I20260925 09:27:27.504210 18802 tablet_bootstrap.cc:492] T 349c4342cf0a44c8aa0dbfc3adef5307 P 4ef88ee04cb149f7982c63065d327f9d: Bootstrap starting.
I20260925 09:27:27.504714 18804 tablet_bootstrap.cc:654] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:27.505930 18802 tablet_bootstrap.cc:654] T 349c4342cf0a44c8aa0dbfc3adef5307 P 4ef88ee04cb149f7982c63065d327f9d: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:27.508363 18803 tablet_bootstrap.cc:492] T 349c4342cf0a44c8aa0dbfc3adef5307 P 18099eba8274427087a5f753f8f86e75: No bootstrap required, opened a new log
I20260925 09:27:27.508538 18803 ts_tablet_manager.cc:1397] T 349c4342cf0a44c8aa0dbfc3adef5307 P 18099eba8274427087a5f753f8f86e75: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260925 09:27:27.509400 18803 raft_consensus.cc:348] T 349c4342cf0a44c8aa0dbfc3adef5307 P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:27.509620 18803 raft_consensus.cc:374] T 349c4342cf0a44c8aa0dbfc3adef5307 P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:27.509713 18803 raft_consensus.cc:729] T 349c4342cf0a44c8aa0dbfc3adef5307 P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18099eba8274427087a5f753f8f86e75, State: Initialized, Role: FOLLOWER
I20260925 09:27:27.510003 18803 consensus_queue.cc:260] T 349c4342cf0a44c8aa0dbfc3adef5307 P 18099eba8274427087a5f753f8f86e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:27.511404 18803 ts_tablet_manager.cc:1428] T 349c4342cf0a44c8aa0dbfc3adef5307 P 18099eba8274427087a5f753f8f86e75: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 09:27:27.516853 18826 tablet_replica.cc:330] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d: stopping tablet replica
I20260925 09:27:27.517436 18826 raft_consensus.cc:2229] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:27:27.518213 18826 raft_consensus.cc:2258] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:27.518997 18827 tablet_replica.cc:330] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75: stopping tablet replica
I20260925 09:27:27.519516 18827 raft_consensus.cc:2229] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:27:27.519896 18802 tablet_bootstrap.cc:492] T 349c4342cf0a44c8aa0dbfc3adef5307 P 4ef88ee04cb149f7982c63065d327f9d: No bootstrap required, opened a new log
I20260925 09:27:27.520032 18802 ts_tablet_manager.cc:1397] T 349c4342cf0a44c8aa0dbfc3adef5307 P 4ef88ee04cb149f7982c63065d327f9d: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.005s
I20260925 09:27:27.520522 18827 raft_consensus.cc:2258] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:27.520900 18802 raft_consensus.cc:348] T 349c4342cf0a44c8aa0dbfc3adef5307 P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:27.521124 18802 raft_consensus.cc:374] T 349c4342cf0a44c8aa0dbfc3adef5307 P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:27.521220 18802 raft_consensus.cc:729] T 349c4342cf0a44c8aa0dbfc3adef5307 P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ef88ee04cb149f7982c63065d327f9d, State: Initialized, Role: FOLLOWER
I20260925 09:27:27.521461 18802 consensus_queue.cc:260] T 349c4342cf0a44c8aa0dbfc3adef5307 P 4ef88ee04cb149f7982c63065d327f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:27.522035 18802 ts_tablet_manager.cc:1428] T 349c4342cf0a44c8aa0dbfc3adef5307 P 4ef88ee04cb149f7982c63065d327f9d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 09:27:27.523721 18804 tablet_bootstrap.cc:492] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a: No bootstrap required, opened a new log
I20260925 09:27:27.523932 18804 ts_tablet_manager.cc:1397] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.007s
I20260925 09:27:27.525033 18804 raft_consensus.cc:348] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:27.525697 18804 raft_consensus.cc:374] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:27.525882 18804 raft_consensus.cc:729] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0627b57c8454d46841f7d512624150a, State: Initialized, Role: FOLLOWER
I20260925 09:27:27.526222 18804 consensus_queue.cc:260] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:27.526918 18804 ts_tablet_manager.cc:1428] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 09:27:27.527040 18827 ts_tablet_manager.cc:1905] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:27.527647 18828 tablet_replica.cc:330] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a: stopping tablet replica
I20260925 09:27:27.527946 18826 ts_tablet_manager.cc:1905] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:27.528070 18828 raft_consensus.cc:2229] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:27:27.530297 18828 raft_consensus.cc:2258] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:27.535084 18826 ts_tablet_manager.cc:1918] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:27:27.535236 18826 log.cc:1196] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/1d2034121a6b47e185712c8bef86655a
I20260925 09:27:27.535408 18828 ts_tablet_manager.cc:1905] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:27.535800 18826 ts_tablet_manager.cc:1939] T 1d2034121a6b47e185712c8bef86655a P 4ef88ee04cb149f7982c63065d327f9d: Deleting consensus metadata
I20260925 09:27:27.537403 18279 catalog_manager.cc:4841] TS 4ef88ee04cb149f7982c63065d327f9d (127.10.114.66:46661): tablet 1d2034121a6b47e185712c8bef86655a (table test-table [id=cb377469bb5947d193e667b6d98feed7]) successfully deleted
I20260925 09:27:27.539990 18828 ts_tablet_manager.cc:1918] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:27:27.540153 18828 log.cc:1196] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/1d2034121a6b47e185712c8bef86655a
I20260925 09:27:27.540580 18828 ts_tablet_manager.cc:1939] T 1d2034121a6b47e185712c8bef86655a P b0627b57c8454d46841f7d512624150a: Deleting consensus metadata
I20260925 09:27:27.540704 18827 ts_tablet_manager.cc:1918] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:27:27.540859 18827 log.cc:1196] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/1d2034121a6b47e185712c8bef86655a
I20260925 09:27:27.541355 18827 ts_tablet_manager.cc:1939] T 1d2034121a6b47e185712c8bef86655a P 18099eba8274427087a5f753f8f86e75: Deleting consensus metadata
I20260925 09:27:27.542212 18278 catalog_manager.cc:4841] TS b0627b57c8454d46841f7d512624150a (127.10.114.67:46043): tablet 1d2034121a6b47e185712c8bef86655a (table test-table [id=cb377469bb5947d193e667b6d98feed7]) successfully deleted
I20260925 09:27:27.542933 18281 catalog_manager.cc:4841] TS 18099eba8274427087a5f753f8f86e75 (127.10.114.65:39449): tablet 1d2034121a6b47e185712c8bef86655a (table test-table [id=cb377469bb5947d193e667b6d98feed7]) successfully deleted
I20260925 09:27:27.619483 18810 raft_consensus.cc:482] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:27:27.619722 18810 raft_consensus.cc:504] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:27.621599 18810 leader_election.cc:288] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18099eba8274427087a5f753f8f86e75 (127.10.114.65:39449), 4ef88ee04cb149f7982c63065d327f9d (127.10.114.66:46661)
I20260925 09:27:27.629231 18800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.622605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:41959 (local address 127.10.114.66:46661)
0925 09:27:27.623095 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:27.623099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:27.623130 (+ 31us) server_negotiation.cc:407] Connection header received
0925 09:27:27.623299 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:27.623306 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:27.623378 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:27.623488 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:27.624104 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.625220 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.627504 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.628328 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.628493 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.628562 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:27.628814 (+ 252us) server_negotiation.cc:299] Negotiation successful
0925 09:27:27.628936 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340}
I20260925 09:27:27.629308 18832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.621948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:46661 (local address 127.10.114.67:41959)
0925 09:27:27.623017 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:27.623035 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:27.623149 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:27.623513 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:27.623520 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:27.623544 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:27.623928 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:27.623940 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.625342 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.625347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:27.627363 (+ 2016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:27.627372 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.628515 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.628521 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:27.628672 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.628695 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 09:27:27.628793 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 09:27:27.628916 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":946,"mutex_wait_us":39,"thread_start_us":835,"threads_started":1}
I20260925 09:27:27.629868 18667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.621904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:39449 (local address 127.10.114.67:59133)
0925 09:27:27.622199 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:27.622225 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:27.622376 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:27.623197 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:27.623228 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:27.623262 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:27:27.623489 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:27.623501 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.624881 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.624886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:27.627082 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:27.627096 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.628759 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.628767 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:27.628966 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.628998 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 09:27:27.629086 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 09:27:27.629211 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":26}
I20260925 09:27:27.630147 18594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "349c4342cf0a44c8aa0dbfc3adef5307" candidate_uuid: "b0627b57c8454d46841f7d512624150a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef88ee04cb149f7982c63065d327f9d" is_pre_election: true
I20260925 09:27:27.630617 18594 raft_consensus.cc:2454] T 349c4342cf0a44c8aa0dbfc3adef5307 P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0627b57c8454d46841f7d512624150a in term 0.
I20260925 09:27:27.630868 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.622102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:59133 (local address 127.10.114.65:39449)
0925 09:27:27.622349 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:27.622355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:27.622376 (+ 21us) server_negotiation.cc:407] Connection header received
0925 09:27:27.622603 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:27.622613 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:27.622733 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:27.622824 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:27:27.623658 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.624728 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.627236 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.628448 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.628599 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.628681 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:27.630307 (+ 1626us) server_negotiation.cc:299] Negotiation successful
0925 09:27:27.630435 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260925 09:27:27.631214 18660 leader_election.cc:302] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ef88ee04cb149f7982c63065d327f9d, b0627b57c8454d46841f7d512624150a; no voters:
I20260925 09:27:27.631562 18810 raft_consensus.cc:2781] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:27:27.631692 18810 raft_consensus.cc:482] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:27:27.631819 18810 raft_consensus.cc:3037] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:27.634228 18450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "349c4342cf0a44c8aa0dbfc3adef5307" candidate_uuid: "b0627b57c8454d46841f7d512624150a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18099eba8274427087a5f753f8f86e75" is_pre_election: true
I20260925 09:27:27.634438 18450 raft_consensus.cc:2454] T 349c4342cf0a44c8aa0dbfc3adef5307 P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0627b57c8454d46841f7d512624150a in term 0.
I20260925 09:27:27.634794 18810 raft_consensus.cc:504] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:27.635422 18810 leader_election.cc:288] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [CANDIDATE]: Term 1 election: Requested vote from peers 18099eba8274427087a5f753f8f86e75 (127.10.114.65:39449), 4ef88ee04cb149f7982c63065d327f9d (127.10.114.66:46661)
I20260925 09:27:27.635790 18450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "349c4342cf0a44c8aa0dbfc3adef5307" candidate_uuid: "b0627b57c8454d46841f7d512624150a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18099eba8274427087a5f753f8f86e75"
I20260925 09:27:27.635974 18450 raft_consensus.cc:3037] T 349c4342cf0a44c8aa0dbfc3adef5307 P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:27.636071 18594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "349c4342cf0a44c8aa0dbfc3adef5307" candidate_uuid: "b0627b57c8454d46841f7d512624150a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef88ee04cb149f7982c63065d327f9d"
I20260925 09:27:27.636245 18594 raft_consensus.cc:3037] T 349c4342cf0a44c8aa0dbfc3adef5307 P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:27.638657 18450 raft_consensus.cc:2454] T 349c4342cf0a44c8aa0dbfc3adef5307 P 18099eba8274427087a5f753f8f86e75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0627b57c8454d46841f7d512624150a in term 1.
I20260925 09:27:27.639103 18662 leader_election.cc:302] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18099eba8274427087a5f753f8f86e75, b0627b57c8454d46841f7d512624150a; no voters:
I20260925 09:27:27.639309 18594 raft_consensus.cc:2454] T 349c4342cf0a44c8aa0dbfc3adef5307 P 4ef88ee04cb149f7982c63065d327f9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0627b57c8454d46841f7d512624150a in term 1.
I20260925 09:27:27.639493 18810 raft_consensus.cc:2781] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:27.640008 18810 raft_consensus.cc:686] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [term 1 LEADER]: Becoming Leader. State: Replica: b0627b57c8454d46841f7d512624150a, State: Running, Role: LEADER
I20260925 09:27:27.640661 18810 consensus_queue.cc:237] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:27.644009 18294 catalog_manager.cc:5495] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a reported cstate change: term changed from 0 to 1, leader changed from <none> to b0627b57c8454d46841f7d512624150a (127.10.114.67). New cstate: current_term: 1 leader_uuid: "b0627b57c8454d46841f7d512624150a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } health_report { overall_health: HEALTHY } } }
Sep 25 09:27:27 dist-test-slave-9pj8 krb5kdc[18249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328445, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.126@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM
I20260925 09:27:27.696318 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.682122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59680 (local address 127.10.114.126:44883)
0925 09:27:27.682317 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:27.682326 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:27.682880 (+ 554us) server_negotiation.cc:407] Connection header received
0925 09:27:27.683041 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:27.683045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:27.683102 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:27.683188 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:27.684619 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.685357 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.687168 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.687550 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.687652 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.691305 (+ 3653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:27.691309 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:27.691312 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:27.691338 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:27.693766 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:27.694403 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:27.694411 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:27.694414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:27.694528 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:27.694861 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:27.694869 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:27.694872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:27.695108 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:27.695254 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:27.695667 (+ 413us) server_negotiation.cc:299] Negotiation successful
0925 09:27:27.695995 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":145,"server-negotiator.queue_time_us":63}
I20260925 09:27:27.696339 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:27.682154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:44883 (local address 127.0.0.1:59680)
0925 09:27:27.682777 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:27.682797 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:27.682938 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:27.683236 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:27.683241 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:27.684153 (+ 912us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:27.684464 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:27.684478 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.685477 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.685483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:27.687045 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:27.687058 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:27.687709 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:27.687718 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:27.687914 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:27.688605 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:27.688634 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:27.691112 (+ 2478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:27.693890 (+ 2778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:27.693903 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:27.693911 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:27.694294 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:27.694633 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:27.694637 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:27.694641 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:27.694766 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:27.695242 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:27.695250 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:27.695406 (+ 156us) client_negotiation.cc:769] Sending connection context
0925 09:27:27.695561 (+ 155us) client_negotiation.cc:240] Negotiation successful
0925 09:27:27.695861 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":308,"threads_started":1}
I20260925 09:27:27.701392 18294 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:59680:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 09:27:27.702382 18294 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:27:27.708981 18569 tablet_service.cc:1467] Processing CreateTablet for tablet 6083f89ef1964b68a73907ee22e7c46b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8555a35598164e53b8d34509fcb10c1b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 09:27:27.708947 18708 tablet_service.cc:1467] Processing CreateTablet for tablet 6083f89ef1964b68a73907ee22e7c46b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8555a35598164e53b8d34509fcb10c1b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 09:27:27.709522 18708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6083f89ef1964b68a73907ee22e7c46b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:27.709523 18569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6083f89ef1964b68a73907ee22e7c46b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:27.716614 18804 tablet_bootstrap.cc:492] T 6083f89ef1964b68a73907ee22e7c46b P b0627b57c8454d46841f7d512624150a: Bootstrap starting.
I20260925 09:27:27.716457 18430 tablet_service.cc:1467] Processing CreateTablet for tablet 6083f89ef1964b68a73907ee22e7c46b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8555a35598164e53b8d34509fcb10c1b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 09:27:27.717036 18430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6083f89ef1964b68a73907ee22e7c46b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:27.718182 18804 tablet_bootstrap.cc:654] T 6083f89ef1964b68a73907ee22e7c46b P b0627b57c8454d46841f7d512624150a: Neither blocks nor log segments found. Creating new log.
W20260925 09:27:27.723919 18640 tablet.cc:2369] T 349c4342cf0a44c8aa0dbfc3adef5307 P 4ef88ee04cb149f7982c63065d327f9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:27:27.724263 18803 tablet_bootstrap.cc:492] T 6083f89ef1964b68a73907ee22e7c46b P 18099eba8274427087a5f753f8f86e75: Bootstrap starting.
I20260925 09:27:27.725919 18803 tablet_bootstrap.cc:654] T 6083f89ef1964b68a73907ee22e7c46b P 18099eba8274427087a5f753f8f86e75: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:27.729816 18802 tablet_bootstrap.cc:492] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d: Bootstrap starting.
I20260925 09:27:27.732091 18802 tablet_bootstrap.cc:654] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:27.735016 18803 tablet_bootstrap.cc:492] T 6083f89ef1964b68a73907ee22e7c46b P 18099eba8274427087a5f753f8f86e75: No bootstrap required, opened a new log
I20260925 09:27:27.735216 18803 ts_tablet_manager.cc:1397] T 6083f89ef1964b68a73907ee22e7c46b P 18099eba8274427087a5f753f8f86e75: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260925 09:27:27.736869 18803 raft_consensus.cc:348] T 6083f89ef1964b68a73907ee22e7c46b P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } }
I20260925 09:27:27.737272 18803 raft_consensus.cc:374] T 6083f89ef1964b68a73907ee22e7c46b P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:27.737396 18803 raft_consensus.cc:729] T 6083f89ef1964b68a73907ee22e7c46b P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18099eba8274427087a5f753f8f86e75, State: Initialized, Role: FOLLOWER
I20260925 09:27:27.737744 18803 consensus_queue.cc:260] T 6083f89ef1964b68a73907ee22e7c46b P 18099eba8274427087a5f753f8f86e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } }
I20260925 09:27:27.738238 18808 tablet_replica.cc:439] T 6083f89ef1964b68a73907ee22e7c46b P 18099eba8274427087a5f753f8f86e75: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } }
I20260925 09:27:27.738579 18808 tablet_replica.cc:442] T 6083f89ef1964b68a73907ee22e7c46b P 18099eba8274427087a5f753f8f86e75: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:27:27.741204 18803 ts_tablet_manager.cc:1428] T 6083f89ef1964b68a73907ee22e7c46b P 18099eba8274427087a5f753f8f86e75: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260925 09:27:27.745328 18802 tablet_bootstrap.cc:492] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d: No bootstrap required, opened a new log
I20260925 09:27:27.745498 18802 ts_tablet_manager.cc:1397] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.004s
I20260925 09:27:27.745859 18804 tablet_bootstrap.cc:492] T 6083f89ef1964b68a73907ee22e7c46b P b0627b57c8454d46841f7d512624150a: No bootstrap required, opened a new log
I20260925 09:27:27.746013 18804 ts_tablet_manager.cc:1397] T 6083f89ef1964b68a73907ee22e7c46b P b0627b57c8454d46841f7d512624150a: Time spent bootstrapping tablet: real 0.029s user 0.009s sys 0.019s
I20260925 09:27:27.746313 18802 raft_consensus.cc:348] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } }
I20260925 09:27:27.746554 18802 raft_consensus.cc:374] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:27.746646 18802 raft_consensus.cc:729] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ef88ee04cb149f7982c63065d327f9d, State: Initialized, Role: FOLLOWER
I20260925 09:27:27.746758 18804 raft_consensus.cc:348] T 6083f89ef1964b68a73907ee22e7c46b P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } }
I20260925 09:27:27.746977 18804 raft_consensus.cc:374] T 6083f89ef1964b68a73907ee22e7c46b P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:27.747044 18804 raft_consensus.cc:729] T 6083f89ef1964b68a73907ee22e7c46b P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0627b57c8454d46841f7d512624150a, State: Initialized, Role: FOLLOWER
I20260925 09:27:27.746954 18802 consensus_queue.cc:260] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } }
I20260925 09:27:27.747310 18804 consensus_queue.cc:260] T 6083f89ef1964b68a73907ee22e7c46b P b0627b57c8454d46841f7d512624150a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } }
I20260925 09:27:27.747395 18813 tablet_replica.cc:439] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } }
I20260925 09:27:27.747689 18813 tablet_replica.cc:442] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:27:27.747686 18810 tablet_replica.cc:439] T 6083f89ef1964b68a73907ee22e7c46b P b0627b57c8454d46841f7d512624150a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } }
I20260925 09:27:27.748603 18810 tablet_replica.cc:442] T 6083f89ef1964b68a73907ee22e7c46b P b0627b57c8454d46841f7d512624150a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:27:27.750128 18802 ts_tablet_manager.cc:1428] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260925 09:27:27.750399 18804 ts_tablet_manager.cc:1428] T 6083f89ef1964b68a73907ee22e7c46b P b0627b57c8454d46841f7d512624150a: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260925 09:27:28.108248 18813 raft_consensus.cc:482] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:27:28.108441 18813 raft_consensus.cc:504] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } }
I20260925 09:27:28.109052 18813 leader_election.cc:288] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0627b57c8454d46841f7d512624150a (127.10.114.67:46043), 18099eba8274427087a5f753f8f86e75 (127.10.114.65:39449)
I20260925 09:27:28.109592 18450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6083f89ef1964b68a73907ee22e7c46b" candidate_uuid: "4ef88ee04cb149f7982c63065d327f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18099eba8274427087a5f753f8f86e75" is_pre_election: true
I20260925 09:27:28.109561 18728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6083f89ef1964b68a73907ee22e7c46b" candidate_uuid: "4ef88ee04cb149f7982c63065d327f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0627b57c8454d46841f7d512624150a" is_pre_election: true
I20260925 09:27:28.109844 18450 raft_consensus.cc:2454] T 6083f89ef1964b68a73907ee22e7c46b P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ef88ee04cb149f7982c63065d327f9d in term 0.
I20260925 09:27:28.109846 18728 raft_consensus.cc:2454] T 6083f89ef1964b68a73907ee22e7c46b P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ef88ee04cb149f7982c63065d327f9d in term 0.
I20260925 09:27:28.110285 18523 leader_election.cc:302] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18099eba8274427087a5f753f8f86e75, 4ef88ee04cb149f7982c63065d327f9d; no voters:
I20260925 09:27:28.110584 18813 raft_consensus.cc:2781] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:27:28.110715 18813 raft_consensus.cc:482] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:27:28.110867 18813 raft_consensus.cc:3037] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:28.112864 18813 raft_consensus.cc:504] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } }
I20260925 09:27:28.113413 18813 leader_election.cc:288] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d [CANDIDATE]: Term 1 election: Requested vote from peers b0627b57c8454d46841f7d512624150a (127.10.114.67:46043), 18099eba8274427087a5f753f8f86e75 (127.10.114.65:39449)
I20260925 09:27:28.113739 18728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6083f89ef1964b68a73907ee22e7c46b" candidate_uuid: "4ef88ee04cb149f7982c63065d327f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0627b57c8454d46841f7d512624150a"
I20260925 09:27:28.113790 18450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6083f89ef1964b68a73907ee22e7c46b" candidate_uuid: "4ef88ee04cb149f7982c63065d327f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18099eba8274427087a5f753f8f86e75"
I20260925 09:27:28.113938 18728 raft_consensus.cc:3037] T 6083f89ef1964b68a73907ee22e7c46b P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:28.113967 18450 raft_consensus.cc:3037] T 6083f89ef1964b68a73907ee22e7c46b P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:28.115918 18450 raft_consensus.cc:2454] T 6083f89ef1964b68a73907ee22e7c46b P 18099eba8274427087a5f753f8f86e75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ef88ee04cb149f7982c63065d327f9d in term 1.
I20260925 09:27:28.115933 18728 raft_consensus.cc:2454] T 6083f89ef1964b68a73907ee22e7c46b P b0627b57c8454d46841f7d512624150a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ef88ee04cb149f7982c63065d327f9d in term 1.
I20260925 09:27:28.116276 18523 leader_election.cc:302] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18099eba8274427087a5f753f8f86e75, 4ef88ee04cb149f7982c63065d327f9d; no voters:
I20260925 09:27:28.116555 18813 raft_consensus.cc:2781] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:28.116895 18813 raft_consensus.cc:686] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d [term 1 LEADER]: Becoming Leader. State: Replica: 4ef88ee04cb149f7982c63065d327f9d, State: Running, Role: LEADER
I20260925 09:27:28.117157 18813 consensus_queue.cc:237] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } }
I20260925 09:27:28.118546 18844 tablet_replica.cc:439] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d: TxnStatusTablet state changed. Reason: New leader 4ef88ee04cb149f7982c63065d327f9d. Latest consensus state: current_term: 1 leader_uuid: "4ef88ee04cb149f7982c63065d327f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } }
I20260925 09:27:28.118816 18844 tablet_replica.cc:442] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d: This TxnStatusTablet replica's current role is: LEADER
I20260925 09:27:28.119675 18846 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 09:27:28.119913 18846 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 09:27:28.120024 18294 catalog_manager.cc:5495] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ef88ee04cb149f7982c63065d327f9d (127.10.114.66). New cstate: current_term: 1 leader_uuid: "4ef88ee04cb149f7982c63065d327f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } health_report { overall_health: UNKNOWN } } }
I20260925 09:27:28.128000 18810 consensus_queue.cc:1035] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [LEADER]: Connected to new peer: Peer: permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:27:28.144728 18833 consensus_queue.cc:1035] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:27:28.186359 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:28.180534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38230 (local address 127.10.114.66:46661)
0925 09:27:28.181298 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:28.181314 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:28.181331 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:27:28.181375 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:28.181378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:28.181437 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:28.181526 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:27:28.181999 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.182766 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.184659 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.185101 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.185203 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:28.185711 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:27:28.185818 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:27:28.185926 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:28.186061 (+ 135us) server_negotiation.cc:299] Negotiation successful
0925 09:27:28.186113 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":530,"threads_started":1}
I20260925 09:27:28.186471 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:28.180487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:46661 (local address 127.0.0.1:38230)
0925 09:27:28.180704 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:28.180722 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:28.180824 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:28.181590 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:28.181596 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:28.181610 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:27:28.181808 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:28.181819 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.182890 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.182897 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:28.184508 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:28.184522 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.185258 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.185264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:28.185464 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:28.185550 (+ 86us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:27:28.185944 (+ 394us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:27:28.185961 (+ 17us) client_negotiation.cc:769] Sending connection context
0925 09:27:28.186043 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 09:27:28.186192 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260925 09:27:28.189960 18728 raft_consensus.cc:1261] T 6083f89ef1964b68a73907ee22e7c46b P b0627b57c8454d46841f7d512624150a [term 1 FOLLOWER]: Refusing update from remote peer 4ef88ee04cb149f7982c63065d327f9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:27:28.189980 18450 raft_consensus.cc:1261] T 6083f89ef1964b68a73907ee22e7c46b P 18099eba8274427087a5f753f8f86e75 [term 1 FOLLOWER]: Refusing update from remote peer 4ef88ee04cb149f7982c63065d327f9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:27:28.190511 18813 consensus_queue.cc:1035] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:27:28.190857 18844 consensus_queue.cc:1035] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:27:28.193665 18808 tablet_replica.cc:439] T 6083f89ef1964b68a73907ee22e7c46b P 18099eba8274427087a5f753f8f86e75: TxnStatusTablet state changed. Reason: New leader 4ef88ee04cb149f7982c63065d327f9d. Latest consensus state: current_term: 1 leader_uuid: "4ef88ee04cb149f7982c63065d327f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } }
I20260925 09:27:28.193984 18808 tablet_replica.cc:442] T 6083f89ef1964b68a73907ee22e7c46b P 18099eba8274427087a5f753f8f86e75: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:27:28.198050 18844 tablet_replica.cc:439] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ef88ee04cb149f7982c63065d327f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } }
I20260925 09:27:28.198359 18844 tablet_replica.cc:442] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d: This TxnStatusTablet replica's current role is: LEADER
I20260925 09:27:28.201743 18808 tablet_replica.cc:439] T 6083f89ef1964b68a73907ee22e7c46b P 18099eba8274427087a5f753f8f86e75: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ef88ee04cb149f7982c63065d327f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } }
I20260925 09:27:28.201973 18808 tablet_replica.cc:442] T 6083f89ef1964b68a73907ee22e7c46b P 18099eba8274427087a5f753f8f86e75: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:27:28.221565 18833 tablet_replica.cc:439] T 6083f89ef1964b68a73907ee22e7c46b P b0627b57c8454d46841f7d512624150a: TxnStatusTablet state changed. Reason: New leader 4ef88ee04cb149f7982c63065d327f9d. Latest consensus state: current_term: 1 leader_uuid: "4ef88ee04cb149f7982c63065d327f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } }
I20260925 09:27:28.221832 18833 tablet_replica.cc:442] T 6083f89ef1964b68a73907ee22e7c46b P b0627b57c8454d46841f7d512624150a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:27:28.223040 18813 tablet_replica.cc:439] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ef88ee04cb149f7982c63065d327f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } }
I20260925 09:27:28.223352 18813 tablet_replica.cc:442] T 6083f89ef1964b68a73907ee22e7c46b P 4ef88ee04cb149f7982c63065d327f9d: This TxnStatusTablet replica's current role is: LEADER
I20260925 09:27:28.237056 18833 tablet_replica.cc:439] T 6083f89ef1964b68a73907ee22e7c46b P b0627b57c8454d46841f7d512624150a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ef88ee04cb149f7982c63065d327f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } } peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } }
I20260925 09:27:28.237318 18833 tablet_replica.cc:442] T 6083f89ef1964b68a73907ee22e7c46b P b0627b57c8454d46841f7d512624150a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:27:28.240252 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:28.206543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49326 (local address 127.10.114.67:46043)
0925 09:27:28.233249 (+ 26706us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:28.233255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:28.233278 (+ 23us) server_negotiation.cc:407] Connection header received
0925 09:27:28.233356 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:28.233362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:28.233441 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:28.234176 (+ 735us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:27:28.234339 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.235593 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.237558 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.238244 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.238783 (+ 539us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:28.238920 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:27:28.239574 (+ 654us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:27:28.239717 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:28.239869 (+ 152us) server_negotiation.cc:299] Negotiation successful
0925 09:27:28.239942 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26587,"thread_start_us":26469,"threads_started":1}
I20260925 09:27:28.254943 18866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:28.249602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:46661 (local address 127.10.114.67:37211)
0925 09:27:28.250223 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:28.250238 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:28.250328 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:28.250633 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:28.250636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:28.250644 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:27:28.250828 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:28.250834 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.252184 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.252187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:28.253190 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:28.253201 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.254081 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.254085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:28.254188 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:28.254229 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:27:28.254562 (+ 333us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:27:28.254568 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 09:27:28.254628 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 09:27:28.254712 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":340,"threads_started":1}
I20260925 09:27:28.255184 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:28.249633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:37211 (local address 127.10.114.66:46661)
0925 09:27:28.249800 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:28.249829 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:28.250304 (+ 475us) server_negotiation.cc:407] Connection header received
0925 09:27:28.250426 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:28.250431 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:28.250505 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:28.250626 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:27:28.250949 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.252066 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.253308 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.253941 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.254059 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:28.254365 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:27:28.254475 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:27:28.254565 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:28.254651 (+ 86us) server_negotiation.cc:299] Negotiation successful
0925 09:27:28.254719 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 09:27:28.289222 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:28.282525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:58715 (local address 127.10.114.67:46043)
0925 09:27:28.282702 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:28.282708 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:28.283333 (+ 625us) server_negotiation.cc:407] Connection header received
0925 09:27:28.283501 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:28.283506 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:28.283579 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:28.283671 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:27:28.284361 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.285507 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.287681 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.288165 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.288254 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:28.288685 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:27:28.288777 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:27:28.288865 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:28.288988 (+ 123us) server_negotiation.cc:299] Negotiation successful
0925 09:27:28.289043 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 09:27:28.289693 18869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:28.282562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:46043 (local address 127.10.114.66:58715)
0925 09:27:28.283223 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:28.283243 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:28.283387 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:28.283842 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:28.283847 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:28.283858 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:27:28.284202 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:28.284213 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.285692 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.285697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:28.287508 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:28.287526 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.288300 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.288308 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:28.288472 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:28.288528 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:27:28.288877 (+ 349us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:27:28.288887 (+ 10us) client_negotiation.cc:769] Sending connection context
0925 09:27:28.288972 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 09:27:28.289109 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":335,"threads_started":1}
I20260925 09:27:28.326040 18294 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:27:28.326320 18294 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:27:28.329433 18294 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842: Sending DeleteTablet for 3 replicas of tablet 349c4342cf0a44c8aa0dbfc3adef5307
I20260925 09:27:28.330132 18430 tablet_service.cc:1514] Processing DeleteTablet for tablet 349c4342cf0a44c8aa0dbfc3adef5307 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:28 UTC) from {username='oryx', principal='oryx/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:52496
I20260925 09:27:28.330437 18569 tablet_service.cc:1514] Processing DeleteTablet for tablet 349c4342cf0a44c8aa0dbfc3adef5307 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:28 UTC) from {username='oryx', principal='oryx/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:38208
I20260925 09:27:28.330744 18708 tablet_service.cc:1514] Processing DeleteTablet for tablet 349c4342cf0a44c8aa0dbfc3adef5307 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:28 UTC) from {username='oryx', principal='oryx/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:49322
I20260925 09:27:28.340876 18874 tablet_replica.cc:330] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a: stopping tablet replica
I20260925 09:27:28.341060 18874 raft_consensus.cc:2229] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:27:28.341615 18874 raft_consensus.cc:2258] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:28.344647 18874 ts_tablet_manager.cc:1905] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:28.348906 18873 tablet_replica.cc:330] T 349c4342cf0a44c8aa0dbfc3adef5307 P 4ef88ee04cb149f7982c63065d327f9d: stopping tablet replica
I20260925 09:27:28.349125 18873 raft_consensus.cc:2229] T 349c4342cf0a44c8aa0dbfc3adef5307 P 4ef88ee04cb149f7982c63065d327f9d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:27:28.349444 18873 raft_consensus.cc:2258] T 349c4342cf0a44c8aa0dbfc3adef5307 P 4ef88ee04cb149f7982c63065d327f9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:28.352361 18873 ts_tablet_manager.cc:1905] T 349c4342cf0a44c8aa0dbfc3adef5307 P 4ef88ee04cb149f7982c63065d327f9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:28.354147 18874 ts_tablet_manager.cc:1918] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:27:28.354291 18874 log.cc:1196] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/349c4342cf0a44c8aa0dbfc3adef5307
I20260925 09:27:28.354799 18874 ts_tablet_manager.cc:1939] T 349c4342cf0a44c8aa0dbfc3adef5307 P b0627b57c8454d46841f7d512624150a: Deleting consensus metadata
I20260925 09:27:28.355974 18278 catalog_manager.cc:4841] TS b0627b57c8454d46841f7d512624150a (127.10.114.67:46043): tablet 349c4342cf0a44c8aa0dbfc3adef5307 (table test-table [id=324022995e56453a92a378ed9bbda99e]) successfully deleted
I20260925 09:27:28.362620 18873 ts_tablet_manager.cc:1918] T 349c4342cf0a44c8aa0dbfc3adef5307 P 4ef88ee04cb149f7982c63065d327f9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:27:28.362782 18873 log.cc:1196] T 349c4342cf0a44c8aa0dbfc3adef5307 P 4ef88ee04cb149f7982c63065d327f9d: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/349c4342cf0a44c8aa0dbfc3adef5307
I20260925 09:27:28.363256 18873 ts_tablet_manager.cc:1939] T 349c4342cf0a44c8aa0dbfc3adef5307 P 4ef88ee04cb149f7982c63065d327f9d: Deleting consensus metadata
I20260925 09:27:28.364382 18279 catalog_manager.cc:4841] TS 4ef88ee04cb149f7982c63065d327f9d (127.10.114.66:46661): tablet 349c4342cf0a44c8aa0dbfc3adef5307 (table test-table [id=324022995e56453a92a378ed9bbda99e]) successfully deleted
I20260925 09:27:28.364949 18872 tablet_replica.cc:330] T 349c4342cf0a44c8aa0dbfc3adef5307 P 18099eba8274427087a5f753f8f86e75: stopping tablet replica
I20260925 09:27:28.365161 18872 raft_consensus.cc:2229] T 349c4342cf0a44c8aa0dbfc3adef5307 P 18099eba8274427087a5f753f8f86e75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:27:28.365424 18872 raft_consensus.cc:2258] T 349c4342cf0a44c8aa0dbfc3adef5307 P 18099eba8274427087a5f753f8f86e75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:28.367381 18872 ts_tablet_manager.cc:1905] T 349c4342cf0a44c8aa0dbfc3adef5307 P 18099eba8274427087a5f753f8f86e75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:28.374915 18872 ts_tablet_manager.cc:1918] T 349c4342cf0a44c8aa0dbfc3adef5307 P 18099eba8274427087a5f753f8f86e75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:27:28.375041 18872 log.cc:1196] T 349c4342cf0a44c8aa0dbfc3adef5307 P 18099eba8274427087a5f753f8f86e75: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/349c4342cf0a44c8aa0dbfc3adef5307
I20260925 09:27:28.375478 18872 ts_tablet_manager.cc:1939] T 349c4342cf0a44c8aa0dbfc3adef5307 P 18099eba8274427087a5f753f8f86e75: Deleting consensus metadata
I20260925 09:27:28.376549 18281 catalog_manager.cc:4841] TS 18099eba8274427087a5f753f8f86e75 (127.10.114.65:39449): tablet 349c4342cf0a44c8aa0dbfc3adef5307 (table test-table [id=324022995e56453a92a378ed9bbda99e]) successfully deleted
Sep 25 09:27:28 dist-test-slave-9pj8 krb5kdc[18249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 09:27:28 dist-test-slave-9pj8 krb5kdc[18249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:28 dist-test-slave-9pj8 krb5kdc[18249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 09:27:28.434444 18881 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:28.406604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59690 (local address 127.10.114.126:44883)
0925 09:27:28.421877 (+ 15273us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:28.421884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:28.422336 (+ 452us) server_negotiation.cc:407] Connection header received
0925 09:27:28.422565 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:28.422571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:28.422663 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:28.422777 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:28.424475 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.425607 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.427551 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.428245 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.428367 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:28.434102 (+ 5735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59690: BlockingRecv error: recv got EOF from 127.0.0.1:59690 (error 108)
Metrics: {"server-negotiator.queue_time_us":15127,"thread_start_us":15600,"threads_started":1}
Sep 25 09:27:28 dist-test-slave-9pj8 krb5kdc[18249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM
I20260925 09:27:28.455847 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:28.438083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59694 (local address 127.10.114.126:44883)
0925 09:27:28.438258 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:28.438263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:28.440243 (+ 1980us) server_negotiation.cc:407] Connection header received
0925 09:27:28.440391 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:28.440396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:28.440465 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:28.440553 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:28.441957 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.442931 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.444593 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.445211 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.445335 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:28.449852 (+ 4517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:28.449858 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:28.449862 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:28.449898 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:28.453116 (+ 3218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:28.453823 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:28.453829 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:28.453832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:28.453974 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:28.454413 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:28.454418 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:28.454421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:28.454740 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:28.454988 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:28.455289 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 09:27:28.455484 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 09:27:28.459388 18294 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:27:28.460151 18294 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:27:28.467706 18430 tablet_service.cc:1467] Processing CreateTablet for tablet 4f5c131dfec743958f090ee369fec3ec (DEFAULT_TABLE table=test-table [id=2239aba3583f461093bdb77bebf5870f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:28.468008 18708 tablet_service.cc:1467] Processing CreateTablet for tablet 4f5c131dfec743958f090ee369fec3ec (DEFAULT_TABLE table=test-table [id=2239aba3583f461093bdb77bebf5870f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:28.468127 18430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f5c131dfec743958f090ee369fec3ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:28.468400 18708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f5c131dfec743958f090ee369fec3ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:28.472137 18569 tablet_service.cc:1467] Processing CreateTablet for tablet 4f5c131dfec743958f090ee369fec3ec (DEFAULT_TABLE table=test-table [id=2239aba3583f461093bdb77bebf5870f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:28.472872 18569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f5c131dfec743958f090ee369fec3ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:28.482879 18888 tablet_bootstrap.cc:492] T 4f5c131dfec743958f090ee369fec3ec P 18099eba8274427087a5f753f8f86e75: Bootstrap starting.
I20260925 09:27:28.484534 18888 tablet_bootstrap.cc:654] T 4f5c131dfec743958f090ee369fec3ec P 18099eba8274427087a5f753f8f86e75: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:28.487051 18888 tablet_bootstrap.cc:492] T 4f5c131dfec743958f090ee369fec3ec P 18099eba8274427087a5f753f8f86e75: No bootstrap required, opened a new log
I20260925 09:27:28.487188 18888 ts_tablet_manager.cc:1397] T 4f5c131dfec743958f090ee369fec3ec P 18099eba8274427087a5f753f8f86e75: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260925 09:27:28.487963 18888 raft_consensus.cc:348] T 4f5c131dfec743958f090ee369fec3ec P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:28.488191 18888 raft_consensus.cc:374] T 4f5c131dfec743958f090ee369fec3ec P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:28.488293 18888 raft_consensus.cc:729] T 4f5c131dfec743958f090ee369fec3ec P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18099eba8274427087a5f753f8f86e75, State: Initialized, Role: FOLLOWER
I20260925 09:27:28.488535 18888 consensus_queue.cc:260] T 4f5c131dfec743958f090ee369fec3ec P 18099eba8274427087a5f753f8f86e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:28.491289 18888 ts_tablet_manager.cc:1428] T 4f5c131dfec743958f090ee369fec3ec P 18099eba8274427087a5f753f8f86e75: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260925 09:27:28.496302 18891 tablet_bootstrap.cc:492] T 4f5c131dfec743958f090ee369fec3ec P b0627b57c8454d46841f7d512624150a: Bootstrap starting.
I20260925 09:27:28.498980 18890 tablet_bootstrap.cc:492] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d: Bootstrap starting.
I20260925 09:27:28.499112 18891 tablet_bootstrap.cc:654] T 4f5c131dfec743958f090ee369fec3ec P b0627b57c8454d46841f7d512624150a: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:28.505491 18891 tablet_bootstrap.cc:492] T 4f5c131dfec743958f090ee369fec3ec P b0627b57c8454d46841f7d512624150a: No bootstrap required, opened a new log
I20260925 09:27:28.506156 18891 ts_tablet_manager.cc:1397] T 4f5c131dfec743958f090ee369fec3ec P b0627b57c8454d46841f7d512624150a: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260925 09:27:28.506999 18891 raft_consensus.cc:348] T 4f5c131dfec743958f090ee369fec3ec P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:28.507233 18891 raft_consensus.cc:374] T 4f5c131dfec743958f090ee369fec3ec P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:28.507328 18891 raft_consensus.cc:729] T 4f5c131dfec743958f090ee369fec3ec P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0627b57c8454d46841f7d512624150a, State: Initialized, Role: FOLLOWER
I20260925 09:27:28.507640 18891 consensus_queue.cc:260] T 4f5c131dfec743958f090ee369fec3ec P b0627b57c8454d46841f7d512624150a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:28.508574 18890 tablet_bootstrap.cc:654] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:28.508677 18891 ts_tablet_manager.cc:1428] T 4f5c131dfec743958f090ee369fec3ec P b0627b57c8454d46841f7d512624150a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 09:27:28.516934 18890 tablet_bootstrap.cc:492] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d: No bootstrap required, opened a new log
I20260925 09:27:28.517103 18890 ts_tablet_manager.cc:1397] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.000s
I20260925 09:27:28.517743 18890 raft_consensus.cc:348] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:28.517957 18890 raft_consensus.cc:374] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:28.518061 18890 raft_consensus.cc:729] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ef88ee04cb149f7982c63065d327f9d, State: Initialized, Role: FOLLOWER
I20260925 09:27:28.518313 18890 consensus_queue.cc:260] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:28.518826 18890 ts_tablet_manager.cc:1428] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 09:27:28.555167 18844 raft_consensus.cc:482] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:27:28.555358 18844 raft_consensus.cc:504] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:28.556023 18844 leader_election.cc:288] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18099eba8274427087a5f753f8f86e75 (127.10.114.65:39449), b0627b57c8454d46841f7d512624150a (127.10.114.67:46043)
I20260925 09:27:28.556448 18450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f5c131dfec743958f090ee369fec3ec" candidate_uuid: "4ef88ee04cb149f7982c63065d327f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18099eba8274427087a5f753f8f86e75" is_pre_election: true
I20260925 09:27:28.556712 18450 raft_consensus.cc:2454] T 4f5c131dfec743958f090ee369fec3ec P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ef88ee04cb149f7982c63065d327f9d in term 0.
I20260925 09:27:28.556679 18727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f5c131dfec743958f090ee369fec3ec" candidate_uuid: "4ef88ee04cb149f7982c63065d327f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0627b57c8454d46841f7d512624150a" is_pre_election: true
I20260925 09:27:28.556957 18727 raft_consensus.cc:2454] T 4f5c131dfec743958f090ee369fec3ec P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ef88ee04cb149f7982c63065d327f9d in term 0.
I20260925 09:27:28.557180 18523 leader_election.cc:302] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18099eba8274427087a5f753f8f86e75, 4ef88ee04cb149f7982c63065d327f9d; no voters:
I20260925 09:27:28.557567 18844 raft_consensus.cc:2781] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:27:28.557679 18844 raft_consensus.cc:482] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:27:28.557787 18844 raft_consensus.cc:3037] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:28.560380 18844 raft_consensus.cc:504] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:28.561219 18450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f5c131dfec743958f090ee369fec3ec" candidate_uuid: "4ef88ee04cb149f7982c63065d327f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18099eba8274427087a5f753f8f86e75"
I20260925 09:27:28.561275 18727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f5c131dfec743958f090ee369fec3ec" candidate_uuid: "4ef88ee04cb149f7982c63065d327f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0627b57c8454d46841f7d512624150a"
I20260925 09:27:28.561419 18450 raft_consensus.cc:3037] T 4f5c131dfec743958f090ee369fec3ec P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:28.561441 18727 raft_consensus.cc:3037] T 4f5c131dfec743958f090ee369fec3ec P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:28.564670 18727 raft_consensus.cc:2454] T 4f5c131dfec743958f090ee369fec3ec P b0627b57c8454d46841f7d512624150a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ef88ee04cb149f7982c63065d327f9d in term 1.
I20260925 09:27:28.565158 18520 leader_election.cc:302] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ef88ee04cb149f7982c63065d327f9d, b0627b57c8454d46841f7d512624150a; no voters:
I20260925 09:27:28.565457 18867 raft_consensus.cc:2781] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:28.565605 18867 raft_consensus.cc:686] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [term 1 LEADER]: Becoming Leader. State: Replica: 4ef88ee04cb149f7982c63065d327f9d, State: Running, Role: LEADER
I20260925 09:27:28.565779 18450 raft_consensus.cc:2454] T 4f5c131dfec743958f090ee369fec3ec P 18099eba8274427087a5f753f8f86e75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ef88ee04cb149f7982c63065d327f9d in term 1.
I20260925 09:27:28.565940 18867 consensus_queue.cc:237] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:28.566550 18844 leader_election.cc:288] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [CANDIDATE]: Term 1 election: Requested vote from peers 18099eba8274427087a5f753f8f86e75 (127.10.114.65:39449), b0627b57c8454d46841f7d512624150a (127.10.114.67:46043)
I20260925 09:27:28.569067 18294 catalog_manager.cc:5495] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ef88ee04cb149f7982c63065d327f9d (127.10.114.66). New cstate: current_term: 1 leader_uuid: "4ef88ee04cb149f7982c63065d327f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } health_report { overall_health: UNKNOWN } } }
I20260925 09:27:28.588250 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:28.582862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38244 (local address 127.10.114.66:46661)
0925 09:27:28.583063 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:28.583071 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:28.583226 (+ 155us) server_negotiation.cc:407] Connection header received
0925 09:27:28.583403 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:28.583406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:28.583466 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:28.583568 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:27:28.583967 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.584880 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.586455 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.586912 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.587045 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:28.587437 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:27:28.587544 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:27:28.587632 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:28.587766 (+ 134us) server_negotiation.cc:299] Negotiation successful
0925 09:27:28.587874 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260925 09:27:28.590761 18450 raft_consensus.cc:1261] T 4f5c131dfec743958f090ee369fec3ec P 18099eba8274427087a5f753f8f86e75 [term 1 FOLLOWER]: Refusing update from remote peer 4ef88ee04cb149f7982c63065d327f9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:27:28.590746 18727 raft_consensus.cc:1261] T 4f5c131dfec743958f090ee369fec3ec P b0627b57c8454d46841f7d512624150a [term 1 FOLLOWER]: Refusing update from remote peer 4ef88ee04cb149f7982c63065d327f9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:27:28.591320 18844 consensus_queue.cc:1035] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:27:28.591836 18844 consensus_queue.cc:1035] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:27:28.617036 18292 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:27:28.617347 18292 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:27:28.620790 18292 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842: Sending DeleteTablet for 3 replicas of tablet 4f5c131dfec743958f090ee369fec3ec
I20260925 09:27:28.621702 18430 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f5c131dfec743958f090ee369fec3ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:28 UTC) from {username='oryx', principal='oryx/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:52496
I20260925 09:27:28.621702 18708 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f5c131dfec743958f090ee369fec3ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:28 UTC) from {username='oryx', principal='oryx/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:49322
I20260925 09:27:28.621969 18874 tablet_replica.cc:330] T 4f5c131dfec743958f090ee369fec3ec P b0627b57c8454d46841f7d512624150a: stopping tablet replica
I20260925 09:27:28.622187 18874 raft_consensus.cc:2229] T 4f5c131dfec743958f090ee369fec3ec P b0627b57c8454d46841f7d512624150a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:27:28.622378 18874 raft_consensus.cc:2258] T 4f5c131dfec743958f090ee369fec3ec P b0627b57c8454d46841f7d512624150a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:28.622424 18569 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f5c131dfec743958f090ee369fec3ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:28 UTC) from {username='oryx', principal='oryx/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:38208
I20260925 09:27:28.622561 18568 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f5c131dfec743958f090ee369fec3ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:28 UTC) from {username='oryx', principal='oryx/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:38208
I20260925 09:27:28.622634 18872 tablet_replica.cc:330] T 4f5c131dfec743958f090ee369fec3ec P 18099eba8274427087a5f753f8f86e75: stopping tablet replica
I20260925 09:27:28.622822 18872 raft_consensus.cc:2229] T 4f5c131dfec743958f090ee369fec3ec P 18099eba8274427087a5f753f8f86e75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:27:28.623013 18872 raft_consensus.cc:2258] T 4f5c131dfec743958f090ee369fec3ec P 18099eba8274427087a5f753f8f86e75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:28.623425 18873 tablet_replica.cc:330] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d: stopping tablet replica
I20260925 09:27:28.623610 18873 raft_consensus.cc:2229] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:27:28.623914 18873 raft_consensus.cc:2258] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:28.624039 18872 ts_tablet_manager.cc:1905] T 4f5c131dfec743958f090ee369fec3ec P 18099eba8274427087a5f753f8f86e75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:28.624940 18292 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:27:28.625687 18292 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:27:28.630584 18872 ts_tablet_manager.cc:1918] T 4f5c131dfec743958f090ee369fec3ec P 18099eba8274427087a5f753f8f86e75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:27:28.630770 18872 log.cc:1196] T 4f5c131dfec743958f090ee369fec3ec P 18099eba8274427087a5f753f8f86e75: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/4f5c131dfec743958f090ee369fec3ec
I20260925 09:27:28.631297 18872 ts_tablet_manager.cc:1939] T 4f5c131dfec743958f090ee369fec3ec P 18099eba8274427087a5f753f8f86e75: Deleting consensus metadata
I20260925 09:27:28.632380 18281 catalog_manager.cc:4841] TS 18099eba8274427087a5f753f8f86e75 (127.10.114.65:39449): tablet 4f5c131dfec743958f090ee369fec3ec (table test-table [id=2239aba3583f461093bdb77bebf5870f]) successfully deleted
I20260925 09:27:28.632697 18430 tablet_service.cc:1467] Processing CreateTablet for tablet 9692b5eb1987469cbdf9667f5efd4977 (DEFAULT_TABLE table=test-table [id=369d047707474a7198832aa77eb47b22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:28.633056 18430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9692b5eb1987469cbdf9667f5efd4977. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:28.633001 18708 tablet_service.cc:1467] Processing CreateTablet for tablet 9692b5eb1987469cbdf9667f5efd4977 (DEFAULT_TABLE table=test-table [id=369d047707474a7198832aa77eb47b22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:28.633358 18708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9692b5eb1987469cbdf9667f5efd4977. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:28.636592 18568 tablet_service.cc:1467] Processing CreateTablet for tablet 9692b5eb1987469cbdf9667f5efd4977 (DEFAULT_TABLE table=test-table [id=369d047707474a7198832aa77eb47b22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:27:28.637128 18568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9692b5eb1987469cbdf9667f5efd4977. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:28.639931 18888 tablet_bootstrap.cc:492] T 9692b5eb1987469cbdf9667f5efd4977 P 18099eba8274427087a5f753f8f86e75: Bootstrap starting.
I20260925 09:27:28.640314 18874 ts_tablet_manager.cc:1905] T 4f5c131dfec743958f090ee369fec3ec P b0627b57c8454d46841f7d512624150a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:28.641464 18888 tablet_bootstrap.cc:654] T 9692b5eb1987469cbdf9667f5efd4977 P 18099eba8274427087a5f753f8f86e75: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:28.646833 18874 ts_tablet_manager.cc:1918] T 4f5c131dfec743958f090ee369fec3ec P b0627b57c8454d46841f7d512624150a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:27:28.646970 18874 log.cc:1196] T 4f5c131dfec743958f090ee369fec3ec P b0627b57c8454d46841f7d512624150a: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/4f5c131dfec743958f090ee369fec3ec
I20260925 09:27:28.647447 18874 ts_tablet_manager.cc:1939] T 4f5c131dfec743958f090ee369fec3ec P b0627b57c8454d46841f7d512624150a: Deleting consensus metadata
I20260925 09:27:28.648578 18278 catalog_manager.cc:4841] TS b0627b57c8454d46841f7d512624150a (127.10.114.67:46043): tablet 4f5c131dfec743958f090ee369fec3ec (table test-table [id=2239aba3583f461093bdb77bebf5870f]) successfully deleted
I20260925 09:27:28.649785 18890 tablet_bootstrap.cc:492] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d: Bootstrap starting.
I20260925 09:27:28.651409 18890 tablet_bootstrap.cc:654] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:28.655216 18873 ts_tablet_manager.cc:1905] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:28.665796 18891 tablet_bootstrap.cc:492] T 9692b5eb1987469cbdf9667f5efd4977 P b0627b57c8454d46841f7d512624150a: Bootstrap starting.
I20260925 09:27:28.668089 18891 tablet_bootstrap.cc:654] T 9692b5eb1987469cbdf9667f5efd4977 P b0627b57c8454d46841f7d512624150a: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:28.670821 18891 tablet_bootstrap.cc:492] T 9692b5eb1987469cbdf9667f5efd4977 P b0627b57c8454d46841f7d512624150a: No bootstrap required, opened a new log
I20260925 09:27:28.670979 18891 ts_tablet_manager.cc:1397] T 9692b5eb1987469cbdf9667f5efd4977 P b0627b57c8454d46841f7d512624150a: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.003s
I20260925 09:27:28.671635 18891 raft_consensus.cc:348] T 9692b5eb1987469cbdf9667f5efd4977 P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:28.671891 18891 raft_consensus.cc:374] T 9692b5eb1987469cbdf9667f5efd4977 P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:28.671973 18891 raft_consensus.cc:729] T 9692b5eb1987469cbdf9667f5efd4977 P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0627b57c8454d46841f7d512624150a, State: Initialized, Role: FOLLOWER
I20260925 09:27:28.672196 18891 consensus_queue.cc:260] T 9692b5eb1987469cbdf9667f5efd4977 P b0627b57c8454d46841f7d512624150a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:28.672757 18891 ts_tablet_manager.cc:1428] T 9692b5eb1987469cbdf9667f5efd4977 P b0627b57c8454d46841f7d512624150a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 09:27:28.676553 18873 ts_tablet_manager.cc:1918] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:27:28.677151 18873 log.cc:1196] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/4f5c131dfec743958f090ee369fec3ec
I20260925 09:27:28.677871 18873 ts_tablet_manager.cc:1939] T 4f5c131dfec743958f090ee369fec3ec P 4ef88ee04cb149f7982c63065d327f9d: Deleting consensus metadata
I20260925 09:27:28.680161 18279 catalog_manager.cc:4841] TS 4ef88ee04cb149f7982c63065d327f9d (127.10.114.66:46661): tablet 4f5c131dfec743958f090ee369fec3ec (table test-table [id=2239aba3583f461093bdb77bebf5870f]) successfully deleted
W20260925 09:27:28.680887 18279 catalog_manager.cc:4808] TS 4ef88ee04cb149f7982c63065d327f9d (127.10.114.66:46661): delete failed for tablet 4f5c131dfec743958f090ee369fec3ec because the tablet was not found. No further retry: Not found: Tablet not found: 4f5c131dfec743958f090ee369fec3ec
I20260925 09:27:28.681653 18888 tablet_bootstrap.cc:492] T 9692b5eb1987469cbdf9667f5efd4977 P 18099eba8274427087a5f753f8f86e75: No bootstrap required, opened a new log
I20260925 09:27:28.681830 18888 ts_tablet_manager.cc:1397] T 9692b5eb1987469cbdf9667f5efd4977 P 18099eba8274427087a5f753f8f86e75: Time spent bootstrapping tablet: real 0.042s user 0.003s sys 0.006s
I20260925 09:27:28.682565 18888 raft_consensus.cc:348] T 9692b5eb1987469cbdf9667f5efd4977 P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:28.682819 18888 raft_consensus.cc:374] T 9692b5eb1987469cbdf9667f5efd4977 P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:28.682896 18888 raft_consensus.cc:729] T 9692b5eb1987469cbdf9667f5efd4977 P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18099eba8274427087a5f753f8f86e75, State: Initialized, Role: FOLLOWER
I20260925 09:27:28.683120 18888 consensus_queue.cc:260] T 9692b5eb1987469cbdf9667f5efd4977 P 18099eba8274427087a5f753f8f86e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:28.683636 18888 ts_tablet_manager.cc:1428] T 9692b5eb1987469cbdf9667f5efd4977 P 18099eba8274427087a5f753f8f86e75: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260925 09:27:28.684893 18890 tablet_bootstrap.cc:492] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d: No bootstrap required, opened a new log
I20260925 09:27:28.685062 18890 ts_tablet_manager.cc:1397] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d: Time spent bootstrapping tablet: real 0.035s user 0.008s sys 0.025s
I20260925 09:27:28.685845 18890 raft_consensus.cc:348] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:28.686080 18890 raft_consensus.cc:374] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:28.686185 18890 raft_consensus.cc:729] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ef88ee04cb149f7982c63065d327f9d, State: Initialized, Role: FOLLOWER
I20260925 09:27:28.686416 18890 consensus_queue.cc:260] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:28.686970 18890 ts_tablet_manager.cc:1428] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260925 09:27:28.864013 18867 raft_consensus.cc:482] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:27:28.864200 18867 raft_consensus.cc:504] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:28.864786 18867 leader_election.cc:288] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18099eba8274427087a5f753f8f86e75 (127.10.114.65:39449), b0627b57c8454d46841f7d512624150a (127.10.114.67:46043)
I20260925 09:27:28.865316 18450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9692b5eb1987469cbdf9667f5efd4977" candidate_uuid: "4ef88ee04cb149f7982c63065d327f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18099eba8274427087a5f753f8f86e75" is_pre_election: true
I20260925 09:27:28.865442 18727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9692b5eb1987469cbdf9667f5efd4977" candidate_uuid: "4ef88ee04cb149f7982c63065d327f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0627b57c8454d46841f7d512624150a" is_pre_election: true
I20260925 09:27:28.865567 18450 raft_consensus.cc:2454] T 9692b5eb1987469cbdf9667f5efd4977 P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ef88ee04cb149f7982c63065d327f9d in term 0.
I20260925 09:27:28.865748 18727 raft_consensus.cc:2454] T 9692b5eb1987469cbdf9667f5efd4977 P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ef88ee04cb149f7982c63065d327f9d in term 0.
I20260925 09:27:28.865988 18523 leader_election.cc:302] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18099eba8274427087a5f753f8f86e75, 4ef88ee04cb149f7982c63065d327f9d; no voters:
I20260925 09:27:28.866292 18867 raft_consensus.cc:2781] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:27:28.866411 18867 raft_consensus.cc:482] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:27:28.866547 18867 raft_consensus.cc:3037] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:28.868561 18867 raft_consensus.cc:504] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:28.869071 18867 leader_election.cc:288] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [CANDIDATE]: Term 1 election: Requested vote from peers 18099eba8274427087a5f753f8f86e75 (127.10.114.65:39449), b0627b57c8454d46841f7d512624150a (127.10.114.67:46043)
I20260925 09:27:28.869378 18450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9692b5eb1987469cbdf9667f5efd4977" candidate_uuid: "4ef88ee04cb149f7982c63065d327f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18099eba8274427087a5f753f8f86e75"
I20260925 09:27:28.869529 18727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9692b5eb1987469cbdf9667f5efd4977" candidate_uuid: "4ef88ee04cb149f7982c63065d327f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0627b57c8454d46841f7d512624150a"
I20260925 09:27:28.869601 18450 raft_consensus.cc:3037] T 9692b5eb1987469cbdf9667f5efd4977 P 18099eba8274427087a5f753f8f86e75 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:28.869714 18727 raft_consensus.cc:3037] T 9692b5eb1987469cbdf9667f5efd4977 P b0627b57c8454d46841f7d512624150a [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:28.872475 18450 raft_consensus.cc:2454] T 9692b5eb1987469cbdf9667f5efd4977 P 18099eba8274427087a5f753f8f86e75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ef88ee04cb149f7982c63065d327f9d in term 1.
I20260925 09:27:28.872476 18727 raft_consensus.cc:2454] T 9692b5eb1987469cbdf9667f5efd4977 P b0627b57c8454d46841f7d512624150a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ef88ee04cb149f7982c63065d327f9d in term 1.
I20260925 09:27:28.872889 18523 leader_election.cc:302] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18099eba8274427087a5f753f8f86e75, 4ef88ee04cb149f7982c63065d327f9d; no voters:
I20260925 09:27:28.873186 18867 raft_consensus.cc:2781] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:28.873355 18867 raft_consensus.cc:686] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [term 1 LEADER]: Becoming Leader. State: Replica: 4ef88ee04cb149f7982c63065d327f9d, State: Running, Role: LEADER
I20260925 09:27:28.873593 18867 consensus_queue.cc:237] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } }
I20260925 09:27:28.875947 18293 catalog_manager.cc:5495] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ef88ee04cb149f7982c63065d327f9d (127.10.114.66). New cstate: current_term: 1 leader_uuid: "4ef88ee04cb149f7982c63065d327f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ef88ee04cb149f7982c63065d327f9d" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 } health_report { overall_health: UNKNOWN } } }
I20260925 09:27:28.924083 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:28.919312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:49663 (local address 127.10.114.66:46661)
0925 09:27:28.919487 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:28.919499 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:28.919935 (+ 436us) server_negotiation.cc:407] Connection header received
0925 09:27:28.920132 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:28.920137 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:28.920190 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:28.920267 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:27:28.920671 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.921334 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.922545 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.922952 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.923071 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:28.923388 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:27:28.923470 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:27:28.923551 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:28.923668 (+ 117us) server_negotiation.cc:299] Negotiation successful
0925 09:27:28.923731 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":40}
I20260925 09:27:28.924167 18904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:28.919320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:46661 (local address 127.10.114.66:49663)
0925 09:27:28.919848 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:28.919868 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:28.920014 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:28.920286 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:28.920290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:28.920299 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:27:28.920558 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:28.920565 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.921450 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.921455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:28.922413 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:28.922422 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.923071 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.923076 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:28.923230 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:28.923258 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:27:28.923560 (+ 302us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:27:28.923570 (+ 10us) client_negotiation.cc:769] Sending connection context
0925 09:27:28.923653 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 09:27:28.923789 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":1,"thread_start_us":255,"threads_started":1}
I20260925 09:27:28.930490 18450 raft_consensus.cc:1261] T 9692b5eb1987469cbdf9667f5efd4977 P 18099eba8274427087a5f753f8f86e75 [term 1 FOLLOWER]: Refusing update from remote peer 4ef88ee04cb149f7982c63065d327f9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:27:28.930466 18727 raft_consensus.cc:1261] T 9692b5eb1987469cbdf9667f5efd4977 P b0627b57c8454d46841f7d512624150a [term 1 FOLLOWER]: Refusing update from remote peer 4ef88ee04cb149f7982c63065d327f9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:27:28.930953 18813 consensus_queue.cc:1035] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "18099eba8274427087a5f753f8f86e75" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:27:28.931196 18867 consensus_queue.cc:1035] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0627b57c8454d46841f7d512624150a" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 46043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:27:28.969345 18293 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:27:28.969564 18293 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:27:28.971925 18293 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de25c483907244babf8cbcabfaeca842: Sending DeleteTablet for 3 replicas of tablet 9692b5eb1987469cbdf9667f5efd4977
I20260925 09:27:28.972559 18430 tablet_service.cc:1514] Processing DeleteTablet for tablet 9692b5eb1987469cbdf9667f5efd4977 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:28 UTC) from {username='oryx', principal='oryx/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:52496
I20260925 09:27:28.972700 18708 tablet_service.cc:1514] Processing DeleteTablet for tablet 9692b5eb1987469cbdf9667f5efd4977 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:28 UTC) from {username='oryx', principal='oryx/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:49322
I20260925 09:27:28.972838 18872 tablet_replica.cc:330] T 9692b5eb1987469cbdf9667f5efd4977 P 18099eba8274427087a5f753f8f86e75: stopping tablet replica
I20260925 09:27:28.972993 18874 tablet_replica.cc:330] T 9692b5eb1987469cbdf9667f5efd4977 P b0627b57c8454d46841f7d512624150a: stopping tablet replica
I20260925 09:27:28.972992 18872 raft_consensus.cc:2229] T 9692b5eb1987469cbdf9667f5efd4977 P 18099eba8274427087a5f753f8f86e75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:27:28.973039 18569 tablet_service.cc:1514] Processing DeleteTablet for tablet 9692b5eb1987469cbdf9667f5efd4977 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:27:28 UTC) from {username='oryx', principal='oryx/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:38208
I20260925 09:27:28.973251 18872 raft_consensus.cc:2258] T 9692b5eb1987469cbdf9667f5efd4977 P 18099eba8274427087a5f753f8f86e75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:28.973284 18873 tablet_replica.cc:330] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d: stopping tablet replica
I20260925 09:27:28.973394 18874 raft_consensus.cc:2229] T 9692b5eb1987469cbdf9667f5efd4977 P b0627b57c8454d46841f7d512624150a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:27:28.973413 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 18365
I20260925 09:27:28.973443 18873 raft_consensus.cc:2229] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:27:28.973564 18874 raft_consensus.cc:2258] T 9692b5eb1987469cbdf9667f5efd4977 P b0627b57c8454d46841f7d512624150a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:28.973706 18873 raft_consensus.cc:2258] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:27:28.974454 18872 ts_tablet_manager.cc:1905] T 9692b5eb1987469cbdf9667f5efd4977 P 18099eba8274427087a5f753f8f86e75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:28.974785 18874 ts_tablet_manager.cc:1905] T 9692b5eb1987469cbdf9667f5efd4977 P b0627b57c8454d46841f7d512624150a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:28.975863 18873 ts_tablet_manager.cc:1905] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:27:28.980507 18873 ts_tablet_manager.cc:1918] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:27:28.980629 18873 log.cc:1196] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/9692b5eb1987469cbdf9667f5efd4977
I20260925 09:27:28.980844 18874 ts_tablet_manager.cc:1918] T 9692b5eb1987469cbdf9667f5efd4977 P b0627b57c8454d46841f7d512624150a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:27:28.980952 18874 log.cc:1196] T 9692b5eb1987469cbdf9667f5efd4977 P b0627b57c8454d46841f7d512624150a: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/9692b5eb1987469cbdf9667f5efd4977
I20260925 09:27:28.981112 18873 ts_tablet_manager.cc:1939] T 9692b5eb1987469cbdf9667f5efd4977 P 4ef88ee04cb149f7982c63065d327f9d: Deleting consensus metadata
I20260925 09:27:28.981438 18874 ts_tablet_manager.cc:1939] T 9692b5eb1987469cbdf9667f5efd4977 P b0627b57c8454d46841f7d512624150a: Deleting consensus metadata
I20260925 09:27:28.981482 18872 ts_tablet_manager.cc:1918] T 9692b5eb1987469cbdf9667f5efd4977 P 18099eba8274427087a5f753f8f86e75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:27:28.981698 18872 log.cc:1196] T 9692b5eb1987469cbdf9667f5efd4977 P 18099eba8274427087a5f753f8f86e75: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/9692b5eb1987469cbdf9667f5efd4977
I20260925 09:27:28.982199 18872 ts_tablet_manager.cc:1939] T 9692b5eb1987469cbdf9667f5efd4977 P 18099eba8274427087a5f753f8f86e75: Deleting consensus metadata
I20260925 09:27:28.982155 18279 catalog_manager.cc:4841] TS 4ef88ee04cb149f7982c63065d327f9d (127.10.114.66:46661): tablet 9692b5eb1987469cbdf9667f5efd4977 (table test-table [id=369d047707474a7198832aa77eb47b22]) successfully deleted
I20260925 09:27:28.982510 18278 catalog_manager.cc:4841] TS b0627b57c8454d46841f7d512624150a (127.10.114.67:46043): tablet 9692b5eb1987469cbdf9667f5efd4977 (table test-table [id=369d047707474a7198832aa77eb47b22]) successfully deleted
I20260925 09:27:28.983088 18281 catalog_manager.cc:4841] TS 18099eba8274427087a5f753f8f86e75 (127.10.114.65:39449): tablet 9692b5eb1987469cbdf9667f5efd4977 (table test-table [id=369d047707474a7198832aa77eb47b22]) successfully deleted
Sep 25 09:27:28 dist-test-slave-9pj8 krb5kdc[18249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.114.65@KRBTEST.COM
I20260925 09:27:28.996730 18909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:28.974853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52502 (local address 127.10.114.65:39449)
0925 09:27:28.984082 (+ 9229us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:28.984086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:28.984099 (+ 13us) server_negotiation.cc:407] Connection header received
0925 09:27:28.984139 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:28.984142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:28.984192 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:28.984322 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:28.985451 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.986248 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.987292 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:28.987662 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:28.987766 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:28.991564 (+ 3798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:28.991577 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:28.991592 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:28.991622 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:28.994222 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:28.994851 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:28.994858 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:28.994860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:28.994978 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:28.995315 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:28.995322 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:28.995325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:28.995636 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:28.995948 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:28.996227 (+ 279us) server_negotiation.cc:299] Negotiation successful
0925 09:27:28.996418 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9125,"thread_start_us":9033,"threads_started":1}
I20260925 09:27:28.997371 18495 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:29.091029 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 18365
I20260925 09:27:29.108847 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 18504
Sep 25 09:27:29 dist-test-slave-9pj8 krb5kdc[18249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.114.66@KRBTEST.COM
I20260925 09:27:29.123435 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:29.109695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38260 (local address 127.10.114.66:46661)
0925 09:27:29.109856 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:29.109860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:29.109877 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:27:29.109976 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:29.109980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:29.110038 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:29.110123 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:29.111182 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:29.111943 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:29.113232 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:29.113651 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:29.113755 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:29.117832 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:29.117845 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:29.117860 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:29.117897 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:29.120998 (+ 3101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:29.121608 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:29.121615 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:29.121618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:29.121758 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:29.122041 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:29.122044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:29.122046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:29.122426 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:29.122694 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:29.122914 (+ 220us) server_negotiation.cc:299] Negotiation successful
0925 09:27:29.123060 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 09:27:29.124154 18631 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:29.263854 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 18504
I20260925 09:27:29.289412 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 18643
Sep 25 09:27:29 dist-test-slave-9pj8 krb5kdc[18249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.114.67@KRBTEST.COM
I20260925 09:27:29.304544 18912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:29.290097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49328 (local address 127.10.114.67:46043)
0925 09:27:29.290566 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:29.290570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:29.290585 (+ 15us) server_negotiation.cc:407] Connection header received
0925 09:27:29.290637 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:29.290641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:29.290704 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:29.290797 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:29.292215 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:29.292928 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:29.294234 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:29.294661 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:29.294761 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:29.299306 (+ 4545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:29.299328 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:29.299345 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:29.299395 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:29.302007 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:29.302662 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:29.302669 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:29.302671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:29.302801 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:29.303133 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:29.303139 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:29.303145 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:29.303444 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:29.303730 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:29.304013 (+ 283us) server_negotiation.cc:299] Negotiation successful
0925 09:27:29.304234 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":281,"threads_started":1}
I20260925 09:27:29.305164 18773 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:29.426956 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 18643
I20260925 09:27:29.447553 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 18262
I20260925 09:27:29.448801 18334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:29.558935 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 18262
2026-09-25T09:27:29Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5612 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:27:29 dist-test-slave-9pj8 krb5kdc[18917](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:27:29 dist-test-slave-9pj8 krb5kdc[18917](info): set up 2 sockets
Sep 25 09:27:29 dist-test-slave-9pj8 krb5kdc[18917](info): commencing operation
krb5kdc: starting...
W20260925 09:27:30.633401 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.002s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:27:30 dist-test-slave-9pj8 krb5kdc[18917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:27:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:27:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:27:30.855136 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:46497
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:35705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:46497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:31.174578 18930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:31.174834 18930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:31.174983 18930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:31.182081 18930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:27:31.182188 18930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:31.182267 18930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:31.182350 18930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:27:31.182427 18930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:27:31.191565 18930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:35705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:46497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:46497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:31.192870 18930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:31.194126 18930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:31.205235 18935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:31.205214 18936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:31.205417 18938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:31.205947 18930 server_base.cc:1034] running on GCE node
I20260925 09:27:31.206602 18930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:31.207867 18930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:31.209059 18930 hybrid_clock.cc:648] HybridClock initialized: now 1790328451209005 us; error 47 us; skew 500 ppm
I20260925 09:27:31.209362 18930 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-25T09:27:31Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1709 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:27:31 dist-test-slave-9pj8 krb5kdc[18944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:27:31 dist-test-slave-9pj8 krb5kdc[18944](info): set up 2 sockets
Sep 25 09:27:31 dist-test-slave-9pj8 krb5kdc[18944](info): commencing operation
krb5kdc: starting...
W20260925 09:27:32.336952 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:27:32 dist-test-slave-9pj8 krb5kdc[18944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:27:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:27:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:27:32.535547 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:43579
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:43579
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:32.834872 18957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:32.835178 18957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:32.835323 18957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:32.842087 18957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:27:32.842221 18957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:32.842309 18957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:32.842424 18957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:27:32.842496 18957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:27:32.851336 18957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:43579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:43579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:32.852490 18957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:32.853724 18957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:32.865876 18965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:32.865976 18963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:32.865958 18962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:32.871953 18957 server_base.cc:1034] running on GCE node
I20260925 09:27:32.872706 18957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:32.874089 18957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:32.875310 18957 hybrid_clock.cc:648] HybridClock initialized: now 1790328452875264 us; error 50 us; skew 500 ppm
I20260925 09:27:32.875680 18957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:32 dist-test-slave-9pj8 krb5kdc[18944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328452, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:32.878897 18957 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:27:32.880570 18957 webserver.cc:460] Webserver started at http://127.10.114.126:32841/ using document root <none> and password file <none>
I20260925 09:27:32.881347 18957 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:32.881480 18957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:32.881804 18957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:32.884801 18957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "2be4e63158ab4f37bc2dc99af00133ed"
format_stamp: "Formatted at 2026-09-25 09:27:32 on dist-test-slave-9pj8"
server_key: "293f8c9dcdead2f185109c5194c33e5b"
server_key_iv: "a16b62a2e8d599c00703836a35408c51"
server_key_version: "encryptionkey@0"
I20260925 09:27:32.885587 18957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "2be4e63158ab4f37bc2dc99af00133ed"
format_stamp: "Formatted at 2026-09-25 09:27:32 on dist-test-slave-9pj8"
server_key: "293f8c9dcdead2f185109c5194c33e5b"
server_key_iv: "a16b62a2e8d599c00703836a35408c51"
server_key_version: "encryptionkey@0"
I20260925 09:27:32.890796 18957 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:27:32.894385 18972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:32.895658 18957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 09:27:32.895900 18957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "2be4e63158ab4f37bc2dc99af00133ed"
format_stamp: "Formatted at 2026-09-25 09:27:32 on dist-test-slave-9pj8"
server_key: "293f8c9dcdead2f185109c5194c33e5b"
server_key_iv: "a16b62a2e8d599c00703836a35408c51"
server_key_version: "encryptionkey@0"
I20260925 09:27:32.896086 18957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:32.917768 18957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:32.921480 18957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:32.921725 18957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:32.939460 18957 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:43579
I20260925 09:27:32.939481 19023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:43579 every 8 connection(s)
I20260925 09:27:32.940799 18957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:27:32.944401 19024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:32.951190 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 18957
I20260925 09:27:32.951386 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:27:32.952214 10697 external_mini_cluster.cc:1428] Setting key 0315a6b7e7c0f8dbaf3ab67bbee91471
I20260925 09:27:32.955214 19024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed: Bootstrap starting.
I20260925 09:27:32.958515 19024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:32.959888 19024 log.cc:824] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:32.962289 19024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed: No bootstrap required, opened a new log
Sep 25 09:27:32 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:27:32.970880 19024 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be4e63158ab4f37bc2dc99af00133ed" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43579 } }
I20260925 09:27:32.971311 19024 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:32.971405 19024 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2be4e63158ab4f37bc2dc99af00133ed, State: Initialized, Role: FOLLOWER
I20260925 09:27:32.972059 19024 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be4e63158ab4f37bc2dc99af00133ed" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43579 } }
I20260925 09:27:32.972312 19024 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:27:32.972445 19024 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:27:32.972646 19024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:32.974784 19027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:32.954270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44112 (local address 127.10.114.126:43579)
0925 09:27:32.955048 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:32.955058 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:32.955100 (+ 42us) server_negotiation.cc:407] Connection header received
0925 09:27:32.956872 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:32.956893 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:32.957277 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:32.957565 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:32.959475 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:32.960872 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:32.961983 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:32.962663 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:32.962883 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:32.966572 (+ 3689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:32.966586 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:32.966613 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:32.966659 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:32.970405 (+ 3746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:32.971018 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:32.971025 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:32.971032 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:32.971218 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:32.971664 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:32.971671 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:32.971675 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:32.972311 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:32.972609 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:32.973097 (+ 488us) server_negotiation.cc:299] Negotiation successful
0925 09:27:32.973502 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":368,"threads_started":1}
I20260925 09:27:32.975562 19024 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be4e63158ab4f37bc2dc99af00133ed" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43579 } }
I20260925 09:27:32.976086 19024 leader_election.cc:302] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2be4e63158ab4f37bc2dc99af00133ed; no voters:
I20260925 09:27:32.976761 19024 leader_election.cc:288] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:27:32.976943 19029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:32.978755 19024 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:27:32.978761 19029 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [term 1 LEADER]: Becoming Leader. State: Replica: 2be4e63158ab4f37bc2dc99af00133ed, State: Running, Role: LEADER
I20260925 09:27:32.979308 19029 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be4e63158ab4f37bc2dc99af00133ed" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43579 } }
I20260925 09:27:32.982339 19030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2be4e63158ab4f37bc2dc99af00133ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be4e63158ab4f37bc2dc99af00133ed" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43579 } } }
I20260925 09:27:32.982739 19030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:32.983296 19031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2be4e63158ab4f37bc2dc99af00133ed. Latest consensus state: current_term: 1 leader_uuid: "2be4e63158ab4f37bc2dc99af00133ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be4e63158ab4f37bc2dc99af00133ed" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43579 } } }
I20260925 09:27:32.983578 19031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:32.983639 19039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:27:32.991343 19039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:27:33.001979 19039 catalog_manager.cc:1347] Generated new cluster ID: f08c18ad038e4408848c41063e0d63b1
I20260925 09:27:33.002075 19039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:27:33.021983 19039 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:27:33.023281 19039 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:27:33.029614 19039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed: Generated new TSK 0
I20260925 09:27:33.030391 19039 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
I20260925 09:27:33.093056 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43579
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:33.416594 19050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:33.416847 19050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:33.416940 19050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:33.423955 19050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:33.424049 19050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:33.424170 19050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:27:33.433435 19050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:33.434527 19050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:33.436033 19050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:33.449532 19055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:33.449586 19058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:33.449597 19056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:33.450344 19050 server_base.cc:1034] running on GCE node
I20260925 09:27:33.450933 19050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:33.451843 19050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:33.453039 19050 hybrid_clock.cc:648] HybridClock initialized: now 1790328453452996 us; error 41 us; skew 500 ppm
I20260925 09:27:33.453312 19050 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:33 dist-test-slave-9pj8 krb5kdc[18944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328453, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:33.456337 19050 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:27:33.457794 19050 webserver.cc:460] Webserver started at http://127.10.114.65:34601/ using document root <none> and password file <none>
I20260925 09:27:33.458477 19050 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:33.458644 19050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:33.458968 19050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:33.461982 19050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "6a966e20db4d4c9b9815af4597a656a6"
format_stamp: "Formatted at 2026-09-25 09:27:33 on dist-test-slave-9pj8"
server_key: "e8e8b74adcf54920b940379421d472df"
server_key_iv: "a00d7930a692930323010b59ee225c06"
server_key_version: "encryptionkey@0"
I20260925 09:27:33.462798 19050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "6a966e20db4d4c9b9815af4597a656a6"
format_stamp: "Formatted at 2026-09-25 09:27:33 on dist-test-slave-9pj8"
server_key: "e8e8b74adcf54920b940379421d472df"
server_key_iv: "a00d7930a692930323010b59ee225c06"
server_key_version: "encryptionkey@0"
I20260925 09:27:33.467834 19050 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260925 09:27:33.471171 19065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:33.472378 19050 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260925 09:27:33.472536 19050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "6a966e20db4d4c9b9815af4597a656a6"
format_stamp: "Formatted at 2026-09-25 09:27:33 on dist-test-slave-9pj8"
server_key: "e8e8b74adcf54920b940379421d472df"
server_key_iv: "a00d7930a692930323010b59ee225c06"
server_key_version: "encryptionkey@0"
I20260925 09:27:33.472714 19050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:33.503012 19050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:33.507126 19050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:33.507411 19050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:33.508369 19050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:33.509647 19050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:33.509738 19050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:33.509873 19050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:33.509938 19050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:33.541147 19050 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:38345
I20260925 09:27:33.541224 19177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:38345 every 8 connection(s)
I20260925 09:27:33.542403 19050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:27:33.545094 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 19050
I20260925 09:27:33.545284 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:27:33.546108 10697 external_mini_cluster.cc:1428] Setting key c2c29d60f6df630a936a1dbe0bfe58f5
Sep 25 09:27:33 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328453, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:27:33.562222 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:33.544721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:50813 (local address 127.10.114.126:43579)
0925 09:27:33.545234 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:33.545240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:33.546427 (+ 1187us) server_negotiation.cc:407] Connection header received
0925 09:27:33.548621 (+ 2194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:33.548627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:33.548703 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:33.548818 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:33.550206 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:33.550925 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:33.552496 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:33.552872 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:33.552985 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:33.556610 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:33.556615 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:33.556618 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:33.556647 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:33.559080 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:33.560009 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:33.560014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:33.560016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:33.560132 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:33.560588 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:33.560591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:33.560593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:33.560829 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:33.560936 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:33.561831 (+ 895us) server_negotiation.cc:299] Negotiation successful
0925 09:27:33.561979 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":287,"threads_started":1}
I20260925 09:27:33.563517 19181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:33.545489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.65:50813)
0925 09:27:33.546269 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:33.546321 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:33.548380 (+ 2059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:33.549046 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:33.549054 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:33.549523 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:33.550054 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:33.550070 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:33.551082 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:33.551087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:33.552366 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:33.552377 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:33.553077 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:33.553083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:33.553288 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:33.553883 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:33.553903 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:33.556420 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:33.559215 (+ 2795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:33.559226 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:33.559243 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:33.559885 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:33.560250 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:33.560257 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:33.560263 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:33.560482 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:33.560952 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:33.560962 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:33.561378 (+ 416us) client_negotiation.cc:769] Sending connection context
0925 09:27:33.561716 (+ 338us) client_negotiation.cc:240] Negotiation successful
0925 09:27:33.562258 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":292,"threads_started":1}
I20260925 09:27:33.565012 19178 heartbeater.cc:347] Connected to a master server at 127.10.114.126:43579
I20260925 09:27:33.565284 19178 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:33.565943 19178 heartbeater.cc:511] Master 127.10.114.126:43579 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.114.66@KRBTEST.COM; defaulting to no policy
I20260925 09:27:33.567925 18989 ts_manager.cc:194] Registered new tserver with Master: 6a966e20db4d4c9b9815af4597a656a6 (127.10.114.65:38345)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.66@KRBTEST.COM" created.
I20260925 09:27:33.569849 18989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:50813
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
I20260925 09:27:33.594362 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43579
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:33.899443 19184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:33.899716 19184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:33.899895 19184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:33.906870 19184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:33.906999 19184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:33.907182 19184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:27:33.916188 19184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:33.917398 19184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:33.918624 19184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:33.930853 19192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:33.930919 19189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:33.930895 19190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:33.931581 19184 server_base.cc:1034] running on GCE node
I20260925 09:27:33.932238 19184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:33.933012 19184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:33.934206 19184 hybrid_clock.cc:648] HybridClock initialized: now 1790328453934161 us; error 32 us; skew 500 ppm
I20260925 09:27:33.934508 19184 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:33 dist-test-slave-9pj8 krb5kdc[18944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328453, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:33.937601 19184 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:27:33.939184 19184 webserver.cc:460] Webserver started at http://127.10.114.66:37357/ using document root <none> and password file <none>
I20260925 09:27:33.940054 19184 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:33.940181 19184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:33.940524 19184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:33.944192 19184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "846312600483455d831ddd1e1d85e727"
format_stamp: "Formatted at 2026-09-25 09:27:33 on dist-test-slave-9pj8"
server_key: "dc8782f9d1491918e0d0a605d67a1b00"
server_key_iv: "ed864831316f6709ef9e80683150d921"
server_key_version: "encryptionkey@0"
I20260925 09:27:33.945093 19184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "846312600483455d831ddd1e1d85e727"
format_stamp: "Formatted at 2026-09-25 09:27:33 on dist-test-slave-9pj8"
server_key: "dc8782f9d1491918e0d0a605d67a1b00"
server_key_iv: "ed864831316f6709ef9e80683150d921"
server_key_version: "encryptionkey@0"
I20260925 09:27:33.950997 19184 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260925 09:27:33.954999 19199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:33.956511 19184 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:27:33.956668 19184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "846312600483455d831ddd1e1d85e727"
format_stamp: "Formatted at 2026-09-25 09:27:33 on dist-test-slave-9pj8"
server_key: "dc8782f9d1491918e0d0a605d67a1b00"
server_key_iv: "ed864831316f6709ef9e80683150d921"
server_key_version: "encryptionkey@0"
I20260925 09:27:33.956853 19184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:33.975896 19184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:33.980196 19184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:33.980409 19184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:33.981343 19184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:33.982560 19184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:33.982626 19184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:33.982703 19184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:33.982836 19184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260925 09:27:34.014833 19184 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:42377
I20260925 09:27:34.014886 19311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:42377 every 8 connection(s)
I20260925 09:27:34.016122 19184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:27:34.021693 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 19184
I20260925 09:27:34.021852 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:27:34.022293 10697 external_mini_cluster.cc:1428] Setting key f6ada8d3fb633332cafa8c2ffc50312a
Sep 25 09:27:34 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328453, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:27:34.038363 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:34.018729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:48881 (local address 127.10.114.126:43579)
0925 09:27:34.018928 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:34.018934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:34.020170 (+ 1236us) server_negotiation.cc:407] Connection header received
0925 09:27:34.022230 (+ 2060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:34.022237 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:34.022364 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:34.022493 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:34.024356 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:34.025159 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:34.027615 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:34.028105 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:34.028233 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:34.032421 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:34.032427 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:34.032433 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:34.032473 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:34.035153 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:34.036116 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:34.036119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:34.036122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:34.036236 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:34.036715 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:34.036718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:34.036720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:34.036966 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:34.037069 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:34.037941 (+ 872us) server_negotiation.cc:299] Negotiation successful
0925 09:27:34.038136 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 09:27:34.040019 19314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:34.019231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.66:48881)
0925 09:27:34.019986 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:34.020024 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:34.021936 (+ 1912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:34.022768 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:34.022780 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:34.023480 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:34.024199 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:34.024218 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:34.025317 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:34.025325 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:34.027468 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:34.027485 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:34.028348 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:34.028357 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:34.028711 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:34.029447 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:34.029485 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:34.032192 (+ 2707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:34.035287 (+ 3095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:34.035300 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:34.035320 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:34.036002 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:34.036340 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:34.036348 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:34.036355 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:34.036597 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:34.037088 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:34.037100 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:34.037564 (+ 464us) client_negotiation.cc:769] Sending connection context
0925 09:27:34.037870 (+ 306us) client_negotiation.cc:240] Negotiation successful
0925 09:27:34.038464 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":273,"threads_started":1}
I20260925 09:27:34.041431 19312 heartbeater.cc:347] Connected to a master server at 127.10.114.126:43579
I20260925 09:27:34.041759 19312 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:34.042698 19312 heartbeater.cc:511] Master 127.10.114.126:43579 requested a full tablet report, sending...
I20260925 09:27:34.044471 18989 ts_manager.cc:194] Registered new tserver with Master: 846312600483455d831ddd1e1d85e727 (127.10.114.66:42377)
I20260925 09:27:34.045365 18989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:48881
WARNING: no policy specified for kudu/127.10.114.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
I20260925 09:27:34.074812 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43579
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:34.386183 19317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:34.386430 19317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:34.386612 19317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:34.394460 19317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:34.394569 19317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:34.394783 19317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:27:34.404585 19317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:43579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:34.405807 19317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:34.407160 19317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:34.420151 19323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:34.420151 19322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:34.420187 19325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:34.421057 19317 server_base.cc:1034] running on GCE node
I20260925 09:27:34.421640 19317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:34.422479 19317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:34.423683 19317 hybrid_clock.cc:648] HybridClock initialized: now 1790328454423639 us; error 30 us; skew 500 ppm
I20260925 09:27:34.424028 19317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:34 dist-test-slave-9pj8 krb5kdc[18944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328454, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:34.427088 19317 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:27:34.428606 19317 webserver.cc:460] Webserver started at http://127.10.114.67:44091/ using document root <none> and password file <none>
I20260925 09:27:34.429235 19317 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:34.429337 19317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:34.429602 19317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:34.432858 19317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "bdb5a21b9daf488cb51ffb65cf970969"
format_stamp: "Formatted at 2026-09-25 09:27:34 on dist-test-slave-9pj8"
server_key: "5f09418a37d21f833ae1676fa6f4ab0e"
server_key_iv: "4112c1183a10549b57be2ade0eac96db"
server_key_version: "encryptionkey@0"
I20260925 09:27:34.433622 19317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "bdb5a21b9daf488cb51ffb65cf970969"
format_stamp: "Formatted at 2026-09-25 09:27:34 on dist-test-slave-9pj8"
server_key: "5f09418a37d21f833ae1676fa6f4ab0e"
server_key_iv: "4112c1183a10549b57be2ade0eac96db"
server_key_version: "encryptionkey@0"
I20260925 09:27:34.438534 19317 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:27:34.442216 19332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:34.443531 19317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:27:34.443742 19317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "bdb5a21b9daf488cb51ffb65cf970969"
format_stamp: "Formatted at 2026-09-25 09:27:34 on dist-test-slave-9pj8"
server_key: "5f09418a37d21f833ae1676fa6f4ab0e"
server_key_iv: "4112c1183a10549b57be2ade0eac96db"
server_key_version: "encryptionkey@0"
I20260925 09:27:34.443980 19317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:34.475314 19317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:34.479485 19317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:34.479822 19317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:34.480837 19317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:34.482354 19317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:34.482462 19317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:34.482613 19317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:34.482692 19317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:34.526219 19317 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:44669
I20260925 09:27:34.526257 19444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:44669 every 8 connection(s)
I20260925 09:27:34.527601 19317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:27:34.534240 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 19317
I20260925 09:27:34.534412 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:27:34.535053 10697 external_mini_cluster.cc:1428] Setting key 75236ba01df835a910cb4d458cde8124
Sep 25 09:27:34 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328454, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:27:34.549682 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:34.530461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:48263 (local address 127.10.114.126:43579)
0925 09:27:34.530653 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:34.530658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:34.531749 (+ 1091us) server_negotiation.cc:407] Connection header received
0925 09:27:34.533714 (+ 1965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:34.533719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:34.533813 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:34.533905 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:34.536195 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:34.537202 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:34.539003 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:34.539435 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:34.539533 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:34.543737 (+ 4204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:34.543742 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:34.543745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:34.543827 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:34.546483 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:34.547263 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:34.547269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:34.547273 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:34.547414 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:34.547897 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:34.547909 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:34.547912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:34.548171 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:34.548322 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:34.549166 (+ 844us) server_negotiation.cc:299] Negotiation successful
0925 09:27:34.549332 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 09:27:34.550863 19447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:34.530851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.67:48263)
0925 09:27:34.531561 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:34.531622 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:34.533424 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:34.534236 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:34.534247 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:34.535082 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:34.535997 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:34.536021 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:34.537352 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:34.537356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:34.538873 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:34.538884 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:34.539628 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:34.539633 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:34.540008 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:34.540952 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:34.540980 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:34.543523 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:34.546667 (+ 3144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:34.546676 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:34.546690 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:34.547132 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:34.547540 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:34.547546 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:34.547550 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:34.547741 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:34.548309 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:34.548317 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:34.548722 (+ 405us) client_negotiation.cc:769] Sending connection context
0925 09:27:34.549039 (+ 317us) client_negotiation.cc:240] Negotiation successful
0925 09:27:34.549521 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":299,"threads_started":1}
I20260925 09:27:34.552182 19445 heartbeater.cc:347] Connected to a master server at 127.10.114.126:43579
I20260925 09:27:34.552484 19445 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:34.553160 19445 heartbeater.cc:511] Master 127.10.114.126:43579 requested a full tablet report, sending...
I20260925 09:27:34.554627 18989 ts_manager.cc:194] Registered new tserver with Master: bdb5a21b9daf488cb51ffb65cf970969 (127.10.114.67:44669)
I20260925 09:27:34.555442 18989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:48263
I20260925 09:27:34.562376 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:27:34.572703 19178 heartbeater.cc:503] Master 127.10.114.126:43579 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.65.keytab.
Entry for principal oryx/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.65.keytab.
WARNING: no policy specified for oryx/127.10.114.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.114.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.66.keytab.
Entry for principal oryx/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.66.keytab.
WARNING: no policy specified for oryx/127.10.114.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.114.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.67.keytab.
Entry for principal oryx/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.67.keytab.
I20260925 09:27:34.690990 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 19050
Sep 25 09:27:34 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:27:34.706652 19455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:34.692325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52650 (local address 127.10.114.65:38345)
0925 09:27:34.693009 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:34.693017 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:34.693049 (+ 32us) server_negotiation.cc:407] Connection header received
0925 09:27:34.693116 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:34.693123 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:34.693358 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:34.693510 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:34.694780 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:34.695836 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:34.697049 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:34.697638 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:34.697737 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:34.701737 (+ 4000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:34.701748 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:34.701764 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:34.701795 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:34.704468 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:34.705027 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:34.705033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:34.705035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:34.705147 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:34.705441 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:34.705444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:34.705446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:34.705749 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:34.705962 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:34.706178 (+ 216us) server_negotiation.cc:299] Negotiation successful
0925 09:27:34.706364 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":353,"threads_started":1}
I20260925 09:27:34.707981 19173 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:34.788265 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 19050
I20260925 09:27:34.804919 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 19184
Sep 25 09:27:34 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.66@KRBTEST.COM
I20260925 09:27:34.819651 19457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:34.805814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42976 (local address 127.10.114.66:42377)
0925 09:27:34.806324 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:34.806331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:34.806355 (+ 24us) server_negotiation.cc:407] Connection header received
0925 09:27:34.806418 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:34.806424 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:34.806640 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:34.806764 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:34.807938 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:34.808887 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:34.810037 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:34.810589 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:34.810678 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:34.814424 (+ 3746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:34.814434 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:34.814447 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:34.814477 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:34.817288 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:34.817836 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:34.817841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:34.817843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:34.817947 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:34.818240 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:34.818243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:34.818245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:34.818526 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:34.818740 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:34.819073 (+ 333us) server_negotiation.cc:299] Negotiation successful
0925 09:27:34.819351 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":257,"threads_started":1}
I20260925 09:27:34.820964 19307 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:34.901851 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 19184
I20260925 09:27:34.919145 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 19317
Sep 25 09:27:34 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.67@KRBTEST.COM
I20260925 09:27:34.934873 19459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:34.920179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54856 (local address 127.10.114.67:44669)
0925 09:27:34.920760 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:34.920767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:34.920793 (+ 26us) server_negotiation.cc:407] Connection header received
0925 09:27:34.920851 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:34.920856 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:34.921052 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:34.921238 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:34.922513 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:34.923475 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:34.924818 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:34.925327 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:34.925454 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:34.929599 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:34.929610 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:34.929630 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:34.929660 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:34.932278 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:34.932895 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:34.932902 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:34.932904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:34.933019 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:34.933406 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:34.933410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:34.933412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:34.933704 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:34.934025 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:34.934304 (+ 279us) server_negotiation.cc:299] Negotiation successful
0925 09:27:34.934558 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":277,"threads_started":1}
I20260925 09:27:34.936101 19440 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:35.017937 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 19317
I20260925 09:27:35.033679 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 18957
I20260925 09:27:35.034822 19019 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:35.131313 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 18957
I20260925 09:27:35.151125 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:43579
--webserver_interface=127.10.114.126
--webserver_port=32841
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:43579
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:35.463719 19462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:35.464012 19462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:35.464147 19462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:35.470729 19462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:27:35.470813 19462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:35.470875 19462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:35.470932 19462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:27:35.470996 19462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:27:35.480201 19462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:43579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:43579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=32841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:35.481367 19462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:35.482547 19462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:35.493645 19468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:35.493693 19470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:35.493690 19467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:35.494874 19462 server_base.cc:1034] running on GCE node
I20260925 09:27:35.495630 19462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:35.496981 19462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:35.498217 19462 hybrid_clock.cc:648] HybridClock initialized: now 1790328455498161 us; error 53 us; skew 500 ppm
I20260925 09:27:35.498544 19462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:35 dist-test-slave-9pj8 krb5kdc[18944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328455, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:35.501443 19462 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:27:35.502740 19462 webserver.cc:460] Webserver started at http://127.10.114.126:32841/ using document root <none> and password file <none>
I20260925 09:27:35.503374 19462 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:35.503521 19462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:35.508476 19462 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260925 09:27:35.511005 19477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:35.512346 19462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:27:35.512522 19462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "2be4e63158ab4f37bc2dc99af00133ed"
format_stamp: "Formatted at 2026-09-25 09:27:32 on dist-test-slave-9pj8"
server_key: "293f8c9dcdead2f185109c5194c33e5b"
server_key_iv: "a16b62a2e8d599c00703836a35408c51"
server_key_version: "encryptionkey@0"
I20260925 09:27:35.513424 19462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:35.545222 19462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:35.548893 19462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:35.549140 19462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:35.566553 19462 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:43579
I20260925 09:27:35.566598 19528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:43579 every 8 connection(s)
I20260925 09:27:35.567957 19462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:27:35.570195 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 19462
I20260925 09:27:35.571265 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:38345
--local_ip_for_outbound_sockets=127.10.114.65
--tserver_master_addrs=127.10.114.126:43579
--webserver_port=34601
--webserver_interface=127.10.114.65
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.114.65
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:27:35.577250 19529 sys_catalog.cc:263] Verifying existing consensus state
I20260925 09:27:35.581260 19529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed: Bootstrap starting.
I20260925 09:27:35.600482 19529 log.cc:824] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:35.609172 19529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 09:27:35.609736 19529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed: Bootstrap complete.
I20260925 09:27:35.619537 19529 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be4e63158ab4f37bc2dc99af00133ed" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43579 } }
I20260925 09:27:35.620447 19529 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2be4e63158ab4f37bc2dc99af00133ed, State: Initialized, Role: FOLLOWER
I20260925 09:27:35.621110 19529 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be4e63158ab4f37bc2dc99af00133ed" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43579 } }
I20260925 09:27:35.621368 19529 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:27:35.621500 19529 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:27:35.621701 19529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:27:35.625564 19529 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be4e63158ab4f37bc2dc99af00133ed" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43579 } }
I20260925 09:27:35.626063 19529 leader_election.cc:302] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2be4e63158ab4f37bc2dc99af00133ed; no voters:
I20260925 09:27:35.626765 19529 leader_election.cc:288] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 09:27:35.626950 19533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:27:35.628135 19533 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [term 2 LEADER]: Becoming Leader. State: Replica: 2be4e63158ab4f37bc2dc99af00133ed, State: Running, Role: LEADER
I20260925 09:27:35.628715 19533 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be4e63158ab4f37bc2dc99af00133ed" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43579 } }
I20260925 09:27:35.629040 19529 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:27:35.631505 19535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2be4e63158ab4f37bc2dc99af00133ed. Latest consensus state: current_term: 2 leader_uuid: "2be4e63158ab4f37bc2dc99af00133ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be4e63158ab4f37bc2dc99af00133ed" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43579 } } }
I20260925 09:27:35.632002 19535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:35.631505 19534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2be4e63158ab4f37bc2dc99af00133ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be4e63158ab4f37bc2dc99af00133ed" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 43579 } } }
I20260925 09:27:35.632545 19534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:35.635802 19539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:27:35.642802 19539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:27:35.644735 19539 catalog_manager.cc:1259] Loaded cluster ID: f08c18ad038e4408848c41063e0d63b1
I20260925 09:27:35.644909 19539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:27:35.650246 19539 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:27:35.652163 19539 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2be4e63158ab4f37bc2dc99af00133ed: Loaded TSK: 0
I20260925 09:27:35.654021 19539 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 09:27:35.919338 19531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:35.919587 19531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:35.919720 19531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:35.926442 19531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:35.926535 19531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:35.926690 19531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:27:35.936129 19531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.65.keytab
--principal=oryx/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:38345
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=34601
--tserver_master_addrs=127.10.114.126:43579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:35.937448 19531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:35.938660 19531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:35.951058 19556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:35.951027 19558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:35.951857 19555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:35.952157 19531 server_base.cc:1034] running on GCE node
I20260925 09:27:35.952698 19531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:35.953567 19531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:35.954775 19531 hybrid_clock.cc:648] HybridClock initialized: now 1790328455954721 us; error 49 us; skew 500 ppm
I20260925 09:27:35.955061 19531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:35 dist-test-slave-9pj8 krb5kdc[18944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328455, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:35.964174 19531 init.cc:377] Logged in from keytab as oryx/127.10.114.65@KRBTEST.COM (short username oryx)
I20260925 09:27:35.965670 19531 webserver.cc:460] Webserver started at http://127.10.114.65:34601/ using document root <none> and password file <none>
I20260925 09:27:35.966328 19531 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:35.966471 19531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:35.971462 19531 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:27:35.974040 19565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:35.975399 19531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:27:35.975544 19531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "6a966e20db4d4c9b9815af4597a656a6"
format_stamp: "Formatted at 2026-09-25 09:27:33 on dist-test-slave-9pj8"
server_key: "e8e8b74adcf54920b940379421d472df"
server_key_iv: "a00d7930a692930323010b59ee225c06"
server_key_version: "encryptionkey@0"
I20260925 09:27:35.976505 19531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:35.993300 19531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:35.997246 19531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:35.997522 19531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:35.998482 19531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:35.999913 19531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:36.000020 19531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:36.000136 19531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:36.000231 19531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:36.032661 19531 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:38345
I20260925 09:27:36.032716 19677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:38345 every 8 connection(s)
I20260925 09:27:36.034117 19531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:27:36.044209 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 19531
I20260925 09:27:36.044796 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:42377
--local_ip_for_outbound_sockets=127.10.114.66
--tserver_master_addrs=127.10.114.126:43579
--webserver_port=37357
--webserver_interface=127.10.114.66
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.114.66
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:27:36 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:36 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:36.051367 19681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:36.037042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.65:51993)
0925 09:27:36.037809 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:36.037843 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:36.039469 (+ 1626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:36.040802 (+ 1333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:36.040814 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:36.041418 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:36.041936 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:36.041953 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.043480 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.043490 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:36.044818 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:36.044830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.045865 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.045872 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:36.046131 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:36.046788 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:36.046809 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:36.050160 (+ 3351us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":341,"threads_started":1}
W20260925 09:27:36.051505 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:36.036875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:51993 (local address 127.10.114.126:43579)
0925 09:27:36.037616 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:36.037632 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:36.037964 (+ 332us) server_negotiation.cc:407] Connection header received
0925 09:27:36.040018 (+ 2054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:36.040035 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:36.040324 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:36.040662 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:36.042119 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.043321 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.045034 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.045651 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.045847 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:36.050486 (+ 4639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.65:51993: BlockingRecv error: recv got EOF from 127.10.114.65:51993 (error 108)
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":231,"threads_started":1}
W20260925 09:27:36.052349 19678 heartbeater.cc:644] Failed to heartbeat to 127.10.114.126:43579 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.114.126:43579: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Sep 25 09:27:36 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:36 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:36.060616 19681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:36.052905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.65:41183)
0925 09:27:36.053078 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:36.053097 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:36.053195 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:36.053569 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:36.053572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:36.053848 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:36.053962 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:36.053969 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.055377 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.055380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:36.056288 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:36.056298 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.057284 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.057288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:36.057409 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:36.057964 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:36.057980 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:36.060455 (+ 2475us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 09:27:36.060956 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:36.053094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:41183 (local address 127.10.114.126:43579)
0925 09:27:36.053279 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:36.053286 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:36.053307 (+ 21us) server_negotiation.cc:407] Connection header received
0925 09:27:36.053357 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:36.053364 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:36.053437 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:36.053547 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:36.054090 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.055248 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.056416 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.057153 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.057287 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:36.060705 (+ 3418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.65:41183: BlockingRecv error: recv got EOF from 127.10.114.65:41183 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 25 09:27:36 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:36 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:36.069054 19681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:36.061204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.65:33135)
0925 09:27:36.061328 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:36.061344 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:36.061435 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:36.061874 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:36.061877 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:36.062099 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:36.062218 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:36.062226 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.063717 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.063720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:36.064599 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:36.064609 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.065596 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.065600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:36.065740 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:36.066258 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:36.066273 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:36.068786 (+ 2513us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 09:27:36.069095 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:36.061393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:33135 (local address 127.10.114.126:43579)
0925 09:27:36.061565 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:36.061572 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:36.061592 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:27:36.061641 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:36.061648 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:36.061745 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:36.061854 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:36.062347 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.063583 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.064734 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.065470 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.065612 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:36.068936 (+ 3324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.65:33135: BlockingRecv error: recv got EOF from 127.10.114.65:33135 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260925 09:27:36.069495 19678 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 09:27:36.362363 19682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:36.362618 19682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:36.362778 19682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:36.370415 19682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:36.370553 19682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:36.370738 19682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:27:36.380281 19682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.66.keytab
--principal=oryx/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:42377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=37357
--tserver_master_addrs=127.10.114.126:43579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:36.381502 19682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:36.382790 19682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:36.396334 19687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:36.396373 19690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:36.396564 19688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:36.397084 19682 server_base.cc:1034] running on GCE node
I20260925 09:27:36.397629 19682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:36.398448 19682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:36.399677 19682 hybrid_clock.cc:648] HybridClock initialized: now 1790328456399635 us; error 33 us; skew 500 ppm
I20260925 09:27:36.400035 19682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:36 dist-test-slave-9pj8 krb5kdc[18944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328456, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:36.402908 19682 init.cc:377] Logged in from keytab as oryx/127.10.114.66@KRBTEST.COM (short username oryx)
I20260925 09:27:36.404264 19682 webserver.cc:460] Webserver started at http://127.10.114.66:37357/ using document root <none> and password file <none>
I20260925 09:27:36.404922 19682 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:36.405026 19682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:36.409969 19682 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260925 09:27:36.412467 19697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:36.413825 19682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:27:36.413990 19682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "846312600483455d831ddd1e1d85e727"
format_stamp: "Formatted at 2026-09-25 09:27:33 on dist-test-slave-9pj8"
server_key: "dc8782f9d1491918e0d0a605d67a1b00"
server_key_iv: "ed864831316f6709ef9e80683150d921"
server_key_version: "encryptionkey@0"
I20260925 09:27:36.414925 19682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:36.433780 19682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:36.437896 19682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:36.438155 19682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:36.439065 19682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:36.440316 19682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:36.440397 19682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:36.440543 19682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:36.440616 19682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:36.471869 19682 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:42377
I20260925 09:27:36.471942 19809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:42377 every 8 connection(s)
I20260925 09:27:36.473168 19682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:27:36.475835 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 19682
I20260925 09:27:36.476634 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:44669
--local_ip_for_outbound_sockets=127.10.114.67
--tserver_master_addrs=127.10.114.126:43579
--webserver_port=44091
--webserver_interface=127.10.114.67
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.114.67
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:27:36 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:36 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 09:27:36.490249 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:36.475567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:35257 (local address 127.10.114.126:43579)
0925 09:27:36.475723 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:36.475727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:36.477277 (+ 1550us) server_negotiation.cc:407] Connection header received
0925 09:27:36.479029 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:36.479035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:36.479122 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:36.479234 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:36.481224 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.481934 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.483381 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.483833 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.483954 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:36.490068 (+ 6114us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.66:35257: BlockingRecv error: recv got EOF from 127.10.114.66:35257 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 09:27:36.491194 19812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:36.476282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.66:35257)
0925 09:27:36.477120 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:36.477162 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:36.478736 (+ 1574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:36.479473 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:36.479482 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:36.480271 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:36.481055 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:36.481075 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.482090 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.482103 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:36.483244 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:36.483255 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.484093 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.484104 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:36.484439 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:36.485183 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:36.485223 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:36.489943 (+ 4720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":315,"threads_started":1}
W20260925 09:27:36.491746 19810 heartbeater.cc:644] Failed to heartbeat to 127.10.114.126:43579 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.114.126:43579: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Sep 25 09:27:36 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:36 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:36.500465 19812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:36.492325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.66:53805)
0925 09:27:36.492484 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:36.492503 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:36.492594 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:36.492882 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:36.492886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:36.493232 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:36.493373 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:36.493379 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.494299 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.494304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:36.495164 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:36.495173 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.495847 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.495852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:36.496013 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:36.496607 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:36.496624 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:36.500145 (+ 3521us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":44}
W20260925 09:27:36.501235 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:36.492435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:53805 (local address 127.10.114.126:43579)
0925 09:27:36.492608 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:36.492629 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:36.492649 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:27:36.492697 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:36.492702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:36.492765 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:36.492861 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:36.493497 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.494171 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.495300 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.495691 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.495841 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:36.500739 (+ 4898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.66:53805: BlockingRecv error: recv got EOF from 127.10.114.66:53805 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 25 09:27:36 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:36 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:36.510752 19812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:36.502303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.66:51449)
0925 09:27:36.502517 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:36.502562 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:36.502687 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:36.503353 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:36.503358 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:36.503814 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:36.503975 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:36.503985 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.505426 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.505430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:36.506318 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:36.506327 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.507285 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.507289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:36.507422 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:36.508039 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:36.508055 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:36.510569 (+ 2514us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 09:27:36.511059 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:36.502756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:51449 (local address 127.10.114.126:43579)
0925 09:27:36.503054 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:36.503059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:36.503087 (+ 28us) server_negotiation.cc:407] Connection header received
0925 09:27:36.503131 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:36.503136 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:36.503217 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:36.503306 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:36.504130 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.505275 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.506443 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.507135 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.507261 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:36.510828 (+ 3567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.66:51449: BlockingRecv error: recv got EOF from 127.10.114.66:51449 (error 108)
Metrics: {"server-negotiator.queue_time_us":161}
W20260925 09:27:36.511206 19810 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 09:27:36.789754 19813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:36.789989 19813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:36.790076 19813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:36.796835 19813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:36.796924 19813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:36.797026 19813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:27:36.805606 19813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/krb5kdc/oryx_127.10.114.67.keytab
--principal=oryx/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:44669
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=44091
--tserver_master_addrs=127.10.114.126:43579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:36.806746 19813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:36.807940 19813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:36.820314 19818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:36.820263 19819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:36.820356 19821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:36.825052 19813 server_base.cc:1034] running on GCE node
I20260925 09:27:36.825650 19813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:36.826514 19813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:36.827698 19813 hybrid_clock.cc:648] HybridClock initialized: now 1790328456827659 us; error 39 us; skew 500 ppm
I20260925 09:27:36.828006 19813 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:36 dist-test-slave-9pj8 krb5kdc[18944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328456, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:36.831037 19813 init.cc:377] Logged in from keytab as oryx/127.10.114.67@KRBTEST.COM (short username oryx)
I20260925 09:27:36.832468 19813 webserver.cc:460] Webserver started at http://127.10.114.67:44091/ using document root <none> and password file <none>
I20260925 09:27:36.833086 19813 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:36.833225 19813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:36.838135 19813 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260925 09:27:36.840871 19828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:36.842075 19813 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:27:36.842243 19813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "bdb5a21b9daf488cb51ffb65cf970969"
format_stamp: "Formatted at 2026-09-25 09:27:34 on dist-test-slave-9pj8"
server_key: "5f09418a37d21f833ae1676fa6f4ab0e"
server_key_iv: "4112c1183a10549b57be2ade0eac96db"
server_key_version: "encryptionkey@0"
I20260925 09:27:36.843173 19813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:36.873955 19813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:36.878502 19813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:36.878764 19813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:36.879874 19813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:36.881224 19813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:36.881361 19813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:36.881491 19813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:36.881577 19813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:36.913829 19813 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:44669
I20260925 09:27:36.913903 19940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:44669 every 8 connection(s)
I20260925 09:27:36.915141 19813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:27:36.919665 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 19813
Sep 25 09:27:36 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:36 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 09:27:36.933374 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:36.917669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:48845 (local address 127.10.114.126:43579)
0925 09:27:36.917830 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:36.917834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:36.919142 (+ 1308us) server_negotiation.cc:407] Connection header received
0925 09:27:36.921382 (+ 2240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:36.921387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:36.921487 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:36.921595 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:36.923498 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.924548 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.926741 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.927475 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.927594 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:36.933164 (+ 5570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.67:48845: BlockingRecv error: recv got EOF from 127.10.114.67:48845 (error 108)
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":54}
I20260925 09:27:36.934244 19943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:36.918268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.67:48845)
0925 09:27:36.918955 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:36.918991 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:36.921122 (+ 2131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:36.921803 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:36.921811 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:36.922511 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:36.923315 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:36.923357 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.924688 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.924692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:36.926625 (+ 1933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:36.926644 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.927690 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.927700 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:36.928006 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:36.928563 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:36.928589 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:36.933001 (+ 4412us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":276,"threads_started":1}
W20260925 09:27:36.934932 19941 heartbeater.cc:644] Failed to heartbeat to 127.10.114.126:43579 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.114.126:43579: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
I20260925 09:27:36.937974 19945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:36.920601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57922 (local address 127.10.114.126:43579)
0925 09:27:36.921830 (+ 1229us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:36.921834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:36.921846 (+ 12us) server_negotiation.cc:407] Connection header received
0925 09:27:36.921883 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:36.921886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:36.921929 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:36.922020 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:36.923842 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.924824 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.926600 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.927255 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.927399 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:36.930449 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:36.930456 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:36.930470 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:36.930501 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:36.934036 (+ 3535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:36.934752 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:36.934760 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:36.934767 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:36.934951 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:36.935347 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:36.935355 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:36.935362 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:36.936015 (+ 653us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:36.936329 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:36.937276 (+ 947us) server_negotiation.cc:299] Negotiation successful
0925 09:27:36.937728 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1157,"thread_start_us":1055,"threads_started":1}
Sep 25 09:27:36 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:36 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:36.942886 19943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:36.935486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.67:38575)
0925 09:27:36.935702 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:36.935720 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:36.935848 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:36.936174 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:36.936178 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:36.936492 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:36.936657 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:36.936664 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.937619 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.937624 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:36.938680 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:36.938692 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.939429 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.939437 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:36.939596 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:36.940083 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:36.940102 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:36.942718 (+ 2616us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114}
W20260925 09:27:36.943022 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:36.935551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:38575 (local address 127.10.114.126:43579)
0925 09:27:36.935680 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:36.935683 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:36.935829 (+ 146us) server_negotiation.cc:407] Connection header received
0925 09:27:36.935939 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:36.935942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:36.936043 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:36.936113 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:36.936763 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.937500 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.938819 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.939294 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.939381 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:36.942851 (+ 3470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.67:38575: BlockingRecv error: recv got EOF from 127.10.114.67:38575 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 25 09:27:36 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:36 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:36.949993 19943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:36.943507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.67:54865)
0925 09:27:36.943669 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:36.943682 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:36.943810 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:36.944121 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:36.944125 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:36.944387 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:36.944504 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:36.944510 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.945351 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.945358 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:36.946206 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:36.946216 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.946840 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.946845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:36.947034 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:36.947534 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:36.947549 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:36.949744 (+ 2195us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260925 09:27:36.950132 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:36.943636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:54865 (local address 127.10.114.126:43579)
0925 09:27:36.943903 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:36.943907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:36.943926 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:27:36.943956 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:36.943958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:36.944012 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:36.944102 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:36.944622 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.945231 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.946340 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:36.946700 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:36.946831 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:36.949914 (+ 3083us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.67:54865: BlockingRecv error: recv got EOF from 127.10.114.67:54865 (error 108)
Metrics: {"server-negotiator.queue_time_us":163}
W20260925 09:27:36.950297 19941 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 25 09:27:37 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:37 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:37.078209 19946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:37.070153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.65:42257)
0925 09:27:37.070654 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:37.070669 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:37.070821 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:37.071184 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:37.071189 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:37.071654 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:37.071880 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:37.071889 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:37.072786 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:37.072792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:37.073720 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:37.073731 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:37.074420 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:37.074427 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:37.074587 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:37.075187 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:37.075204 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:37.077908 (+ 2704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":279,"threads_started":1}
W20260925 09:27:37.078290 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:37.070175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:42257 (local address 127.10.114.126:43579)
0925 09:27:37.070368 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:37.070377 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:37.070757 (+ 380us) server_negotiation.cc:407] Connection header received
0925 09:27:37.070939 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:37.070945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:37.071015 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:37.071114 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:37.072003 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:37.072662 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:37.073853 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:37.074257 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:37.074389 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:37.078095 (+ 3706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.65:42257: BlockingRecv error: recv got EOF from 127.10.114.65:42257 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 25 09:27:37 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:37 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:37.519740 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:37.511735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.66:47799)
0925 09:27:37.512214 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:37.512228 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:37.512341 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:37.512646 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:37.512649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:37.513055 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:37.513234 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:37.513242 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:37.514198 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:37.514202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:37.515139 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:37.515149 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:37.515953 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:37.515959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:37.516128 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:37.516733 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:37.516750 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:37.519514 (+ 2764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":247,"threads_started":1}
W20260925 09:27:37.519980 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:37.511843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:47799 (local address 127.10.114.126:43579)
0925 09:27:37.512019 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:37.512023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:37.512295 (+ 272us) server_negotiation.cc:407] Connection header received
0925 09:27:37.512428 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:37.512431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:37.512506 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:37.512584 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:37.513345 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:37.514066 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:37.515282 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:37.515728 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:37.515921 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:37.519710 (+ 3789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.66:47799: BlockingRecv error: recv got EOF from 127.10.114.66:47799 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 25 09:27:37 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:37 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:37.960155 19948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:37.950884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.67:58941)
0925 09:27:37.951420 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:37.951450 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:37.951631 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:37.952066 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:37.952072 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:37.952591 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:37.952799 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:37.952809 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:37.953847 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:37.953854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:37.954874 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:37.954886 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:37.955693 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:37.955699 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:37.955919 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:37.956595 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:37.956641 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:37.959831 (+ 3190us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":286,"threads_started":1}
W20260925 09:27:37.960253 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:37.950971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:58941 (local address 127.10.114.126:43579)
0925 09:27:37.951254 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:37.951262 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:37.951539 (+ 277us) server_negotiation.cc:407] Connection header received
0925 09:27:37.951806 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:37.951810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:37.951876 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:37.952003 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:37.952952 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:37.953684 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:37.955045 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:37.955489 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:37.955635 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:37.960019 (+ 4384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.67:58941: BlockingRecv error: recv got EOF from 127.10.114.67:58941 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Sep 25 09:27:38 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:38 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:38.087008 19949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:38.079105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.65:55839)
0925 09:27:38.079691 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:38.079707 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:38.079833 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:38.080113 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:38.080116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:38.080516 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:38.080722 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:38.080734 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:38.081746 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:38.081750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:38.082631 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:38.082640 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:38.083278 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:38.083282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:38.083383 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:38.084146 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:38.084163 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:38.086775 (+ 2612us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":339,"threads_started":1}
W20260925 09:27:38.087224 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:38.079204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:55839 (local address 127.10.114.126:43579)
0925 09:27:38.079354 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:38.079358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:38.079795 (+ 437us) server_negotiation.cc:407] Connection header received
0925 09:27:38.079913 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:38.079916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:38.079978 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:38.080060 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:38.080841 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:38.081632 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:38.082752 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:38.083148 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:38.083245 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:38.086968 (+ 3723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.65:55839: BlockingRecv error: recv got EOF from 127.10.114.65:55839 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 25 09:27:38 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:38 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:38.528450 19950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:38.520803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.66:47647)
0925 09:27:38.521258 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:38.521274 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:38.521378 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:38.521625 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:38.521628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:38.521965 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:38.522118 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:38.522126 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:38.523079 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:38.523083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:38.524047 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:38.524058 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:38.524704 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:38.524709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:38.524819 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:38.525363 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:38.525384 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:38.528129 (+ 2745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":242,"threads_started":1}
W20260925 09:27:38.528518 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:38.520887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:47647 (local address 127.10.114.126:43579)
0925 09:27:38.521114 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:38.521120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:38.521344 (+ 224us) server_negotiation.cc:407] Connection header received
0925 09:27:38.521480 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:38.521484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:38.521541 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:38.521615 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:38.522235 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:38.522950 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:38.524174 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:38.524554 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:38.524642 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:38.528319 (+ 3677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.66:47647: BlockingRecv error: recv got EOF from 127.10.114.66:47647 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Sep 25 09:27:38 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:38 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:38.968901 19951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:38.961118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.67:45203)
0925 09:27:38.961639 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:38.961655 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:38.961773 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:38.962064 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:38.962068 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:38.962428 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:38.962609 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:38.962617 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:38.963522 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:38.963525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:38.964375 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:38.964384 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:38.965061 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:38.965066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:38.965204 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:38.965801 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:38.965817 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:38.968668 (+ 2851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":299,"threads_started":1}
W20260925 09:27:38.969180 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:38.961323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:45203 (local address 127.10.114.126:43579)
0925 09:27:38.961483 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:38.961487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:38.961739 (+ 252us) server_negotiation.cc:407] Connection header received
0925 09:27:38.961866 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:38.961869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:38.961941 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:38.962022 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:38.962724 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:38.963418 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:38.964480 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:38.964915 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:38.965031 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:38.968874 (+ 3843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.67:45203: BlockingRecv error: recv got EOF from 127.10.114.67:45203 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 25 09:27:39 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:39 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:39.096076 19952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:39.087967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.65:58181)
0925 09:27:39.088441 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:39.088455 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:39.088556 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:39.088835 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:39.088837 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:39.089122 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:39.089275 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:39.089282 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:39.090295 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:39.090299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:39.091178 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:39.091189 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:39.091874 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:39.091878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:39.092006 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:39.092561 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:39.092577 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:39.095645 (+ 3068us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":254,"threads_started":1}
W20260925 09:27:39.096215 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:39.088105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:58181 (local address 127.10.114.126:43579)
0925 09:27:39.088268 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:39.088273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:39.088522 (+ 249us) server_negotiation.cc:407] Connection header received
0925 09:27:39.088652 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:39.088655 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:39.088714 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:39.088798 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:39.089406 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:39.090147 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:39.091323 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:39.091736 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:39.091863 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:39.095860 (+ 3997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.65:58181: BlockingRecv error: recv got EOF from 127.10.114.65:58181 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 25 09:27:39 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:39 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:39.536754 19953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:39.529302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.66:37167)
0925 09:27:39.529830 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:39.529848 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:39.529956 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:39.530208 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:39.530211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:39.530551 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:39.530714 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:39.530722 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:39.531717 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:39.531720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:39.532575 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:39.532585 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:39.533165 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:39.533168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:39.533300 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:39.533855 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:39.533871 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:39.536514 (+ 2643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":292,"threads_started":1}
W20260925 09:27:39.537024 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:39.529364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:37167 (local address 127.10.114.126:43579)
0925 09:27:39.529514 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:39.529518 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:39.529925 (+ 407us) server_negotiation.cc:407] Connection header received
0925 09:27:39.530042 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:39.530045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:39.530100 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:39.530169 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:39.530843 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:39.531595 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:39.532684 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:39.533059 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:39.533141 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:39.536718 (+ 3577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.66:37167: BlockingRecv error: recv got EOF from 127.10.114.66:37167 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 25 09:27:39 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:39 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:39.977988 19954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:39.969782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.67:49531)
0925 09:27:39.970332 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:39.970347 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:39.970449 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:39.970775 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:39.970779 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:39.971079 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:39.971243 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:39.971251 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:39.972374 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:39.972378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:39.973217 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:39.973227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:39.974034 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:39.974039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:39.974261 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:39.974909 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:39.974926 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:39.977565 (+ 2639us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":256,"threads_started":1}
W20260925 09:27:39.978011 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:39.969882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:49531 (local address 127.10.114.126:43579)
0925 09:27:39.970079 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:39.970084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:39.970421 (+ 337us) server_negotiation.cc:407] Connection header received
0925 09:27:39.970537 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:39.970541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:39.970648 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:39.970736 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:39.971371 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:39.972261 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:39.973344 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:39.973872 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:39.974022 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:39.977770 (+ 3748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.67:49531: BlockingRecv error: recv got EOF from 127.10.114.67:49531 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 25 09:27:40 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:40 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:40.105434 19955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:40.097140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.65:50149)
0925 09:27:40.097641 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:40.097657 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:40.097771 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:40.098035 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:40.098038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:40.098368 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:40.098505 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:40.098513 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:40.099519 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:40.099523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:40.100608 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:40.100619 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:40.101383 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:40.101387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:40.101503 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:40.102053 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:40.102068 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:40.105117 (+ 3049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":271,"threads_started":1}
W20260925 09:27:40.105578 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:40.097228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:50149 (local address 127.10.114.126:43579)
0925 09:27:40.097410 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:40.097415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:40.097740 (+ 325us) server_negotiation.cc:407] Connection header received
0925 09:27:40.097860 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:40.097863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:40.097920 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:40.097998 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:40.098658 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:40.099399 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:40.100771 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:40.101241 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:40.101346 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:40.105328 (+ 3982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.65:50149: BlockingRecv error: recv got EOF from 127.10.114.65:50149 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 25 09:27:40 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:40 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:40.545825 19956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:40.537605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.66:52429)
0925 09:27:40.538090 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:40.538106 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:40.538271 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:40.538585 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:40.538589 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:40.538992 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:40.539131 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:40.539139 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:40.540178 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:40.540185 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:40.541085 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:40.541095 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:40.541798 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:40.541803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:40.541987 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:40.542609 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:40.542626 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:40.545491 (+ 2865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":263,"threads_started":1}
W20260925 09:27:40.545907 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:40.537657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:52429 (local address 127.10.114.126:43579)
0925 09:27:40.537945 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:40.537952 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:40.538194 (+ 242us) server_negotiation.cc:407] Connection header received
0925 09:27:40.538399 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:40.538403 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:40.538463 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:40.538561 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:40.539290 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:40.540064 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:40.541225 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:40.541646 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:40.541790 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:40.545673 (+ 3883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.66:52429: BlockingRecv error: recv got EOF from 127.10.114.66:52429 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 25 09:27:40 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:40 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:40.990119 19957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:40.979118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.67:58373)
0925 09:27:40.979603 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:40.979617 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:40.979725 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:40.980119 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:40.980122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:40.980533 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:40.980736 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:40.980750 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:40.982045 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:40.982048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:40.982922 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:40.982931 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:40.983548 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:40.983552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:40.983667 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:40.987163 (+ 3496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:40.987188 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:40.989889 (+ 2701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":276,"threads_started":1}
W20260925 09:27:40.990321 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:40.979230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:58373 (local address 127.10.114.126:43579)
0925 09:27:40.979384 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:40.979388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:40.979701 (+ 313us) server_negotiation.cc:407] Connection header received
0925 09:27:40.979866 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:40.979869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:40.979967 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:40.980065 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:40.980906 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:40.981940 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:40.983032 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:40.983396 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:40.983481 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:40.990073 (+ 6592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.67:58373: BlockingRecv error: recv got EOF from 127.10.114.67:58373 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 25 09:27:41 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:41 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:41.114632 19958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:41.106352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.65:54077)
0925 09:27:41.106908 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:41.106924 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:41.107032 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:41.107326 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:41.107329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:41.107712 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:41.107892 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:41.107900 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:41.108870 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:41.108874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:41.110150 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:41.110166 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:41.111156 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:41.111162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:41.111285 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:41.111875 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:41.111891 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:41.114417 (+ 2526us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":282,"threads_started":1}
W20260925 09:27:41.114760 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:41.106555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:54077 (local address 127.10.114.126:43579)
0925 09:27:41.106757 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:41.106762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:41.107000 (+ 238us) server_negotiation.cc:407] Connection header received
0925 09:27:41.107123 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:41.107126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:41.107202 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:41.107280 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:41.108052 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:41.108730 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:41.110283 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:41.110940 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:41.111067 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:41.114575 (+ 3508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.65:54077: BlockingRecv error: recv got EOF from 127.10.114.65:54077 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 25 09:27:41 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:41 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:41.554302 19959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:41.546700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.66:35595)
0925 09:27:41.547229 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:41.547246 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:41.547349 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:41.547646 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:41.547649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:41.548060 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:41.548250 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:41.548258 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:41.549210 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:41.549214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:41.550082 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:41.550092 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:41.550747 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:41.550751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:41.550888 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:41.551531 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:41.551546 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:41.554071 (+ 2525us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":292,"threads_started":1}
W20260925 09:27:41.554461 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:41.546932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:35595 (local address 127.10.114.126:43579)
0925 09:27:41.547111 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:41.547117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:41.547321 (+ 204us) server_negotiation.cc:407] Connection header received
0925 09:27:41.547467 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:41.547471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:41.547528 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:41.547599 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:41.548367 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:41.549076 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:41.550197 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:41.550606 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:41.550729 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:41.554238 (+ 3509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.66:35595: BlockingRecv error: recv got EOF from 127.10.114.66:35595 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 25 09:27:41 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:41 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:41.998865 19960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:41.991118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.67:59097)
0925 09:27:41.991638 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:41.991654 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:41.991763 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:41.992081 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:41.992084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:41.992436 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:41.992579 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:41.992586 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:41.993520 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:41.993524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:41.994410 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:41.994420 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:41.995040 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:41.995043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:41.995149 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:41.995739 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:41.995755 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:41.998517 (+ 2762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":273,"threads_started":1}
W20260925 09:27:41.998901 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:41.991189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:59097 (local address 127.10.114.126:43579)
0925 09:27:41.991373 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:41.991377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:41.991756 (+ 379us) server_negotiation.cc:407] Connection header received
0925 09:27:41.991913 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:41.991917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:41.991975 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:41.992062 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:41.992705 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:41.993396 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:41.994531 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:41.994928 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:41.995015 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:41.998679 (+ 3664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.67:59097: BlockingRecv error: recv got EOF from 127.10.114.67:59097 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 25 09:27:42 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:42 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:42.123344 19961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:42.115449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.65:36477)
0925 09:27:42.115977 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:42.116008 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:42.116133 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:42.116472 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:42.116475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:42.116998 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:42.117147 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:42.117156 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:42.118232 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:42.118236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:42.119130 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:42.119141 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:42.119845 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:42.119849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:42.119968 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:42.120562 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:42.120578 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:42.123064 (+ 2486us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":249,"threads_started":1}
W20260925 09:27:42.123466 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:42.115541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:36477 (local address 127.10.114.126:43579)
0925 09:27:42.115826 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:42.115833 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:42.116099 (+ 266us) server_negotiation.cc:407] Connection header received
0925 09:27:42.116246 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:42.116249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:42.116329 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:42.116402 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:42.117300 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:42.118117 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:42.119251 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:42.119673 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:42.119766 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:42.123246 (+ 3480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.65:36477: BlockingRecv error: recv got EOF from 127.10.114.65:36477 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Sep 25 09:27:42 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:42 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:42.563601 19962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:42.555269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.66:39141)
0925 09:27:42.555818 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:42.555853 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:42.555968 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:42.556334 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:42.556338 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:42.556875 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:42.557072 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:42.557084 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:42.558309 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:42.558315 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:42.559189 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:42.559199 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:42.559869 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:42.559876 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:42.560041 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:42.560649 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:42.560669 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:42.563408 (+ 2739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":284,"threads_started":1}
W20260925 09:27:42.563953 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:42.555355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:39141 (local address 127.10.114.126:43579)
0925 09:27:42.555526 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:42.555531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:42.555941 (+ 410us) server_negotiation.cc:407] Connection header received
0925 09:27:42.556079 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:42.556084 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:42.556185 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:42.556299 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:42.557254 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:42.558171 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:42.559322 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:42.559705 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:42.559854 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:42.563627 (+ 3773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.66:39141: BlockingRecv error: recv got EOF from 127.10.114.66:39141 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 25 09:27:43 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:43 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:43.008270 19963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:42.999915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.67:41313)
0925 09:27:43.000397 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:43.000411 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:43.000550 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:43.000890 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:43.000896 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:43.001394 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:43.001535 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:43.001543 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:43.002491 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:43.002496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:43.003356 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:43.003367 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:43.004189 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:43.004195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:43.004363 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:43.004958 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:43.004977 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:43.007955 (+ 2978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":277,"threads_started":1}
W20260925 09:27:43.008368 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:42.999914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:41313 (local address 127.10.114.126:43579)
0925 09:27:43.000100 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:43.000107 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:43.000484 (+ 377us) server_negotiation.cc:407] Connection header received
0925 09:27:43.000692 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:43.000696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:43.000754 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:43.000855 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:43.001664 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:43.002363 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:43.003509 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:43.004038 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:43.004160 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:43.008133 (+ 3973us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.67:41313: BlockingRecv error: recv got EOF from 127.10.114.67:41313 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 25 09:27:43 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:43 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:43.132680 19964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:43.124213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.65:36533)
0925 09:27:43.124793 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:43.124810 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:43.124918 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:43.125192 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:43.125195 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:43.125578 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:43.125752 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:43.125760 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:43.127055 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:43.127063 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:43.128162 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:43.128176 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:43.128795 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:43.128799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:43.128903 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:43.129557 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:43.129591 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:43.132419 (+ 2828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":286,"threads_started":1}
W20260925 09:27:43.132766 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:43.124366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:36533 (local address 127.10.114.126:43579)
0925 09:27:43.124544 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:43.124548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:43.124882 (+ 334us) server_negotiation.cc:407] Connection header received
0925 09:27:43.125013 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:43.125017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:43.125077 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:43.125156 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:43.125925 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:43.126894 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:43.128297 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:43.128678 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:43.128762 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:43.132591 (+ 3829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.65:36533: BlockingRecv error: recv got EOF from 127.10.114.65:36533 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 25 09:27:43 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:43 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:43.574342 19965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:43.564594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.66:44297)
0925 09:27:43.565191 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:43.565213 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:43.565324 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:43.565646 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:43.565649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:43.566022 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:43.566171 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:43.566178 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:43.567705 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:43.567709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:43.568764 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:43.568777 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:43.569786 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:43.569791 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:43.569898 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:43.570488 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:43.570505 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:43.573938 (+ 3433us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":268,"threads_started":1}
W20260925 09:27:43.574362 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:43.564676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:44297 (local address 127.10.114.126:43579)
0925 09:27:43.564912 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:43.564917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:43.565313 (+ 396us) server_negotiation.cc:407] Connection header received
0925 09:27:43.565431 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:43.565436 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:43.565503 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:43.565614 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:43.566319 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:43.567524 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:43.568935 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:43.569635 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:43.569763 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:43.574129 (+ 4366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.66:44297: BlockingRecv error: recv got EOF from 127.10.114.66:44297 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
Sep 25 09:27:44 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:44 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 09:27:44.017706 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:44.009367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:39763 (local address 127.10.114.126:43579)
0925 09:27:44.009537 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:44.009541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:44.009834 (+ 293us) server_negotiation.cc:407] Connection header received
0925 09:27:44.010006 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:44.010010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:44.010072 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:44.010156 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:44.010898 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:44.011610 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:44.012822 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:44.013238 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:44.013335 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:44.017448 (+ 4113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.67:39763: BlockingRecv error: recv got EOF from 127.10.114.67:39763 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:27:44.017690 19966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:44.009216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.67:39763)
0925 09:27:44.009743 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:44.009759 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:44.009865 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:44.010193 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:44.010196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:44.010609 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:44.010761 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:44.010768 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:44.011744 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:44.011748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:44.012696 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:44.012707 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:44.013369 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:44.013373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:44.013479 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:44.014032 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:44.014048 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:44.017270 (+ 3222us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":249,"threads_started":1}
Sep 25 09:27:44 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:44 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:44.142464 19967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:44.133537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.65:38241)
0925 09:27:44.134235 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:44.134254 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:44.134457 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:44.134883 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:44.134889 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:44.135435 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:44.135606 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:44.135616 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:44.136955 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:44.136962 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:44.137868 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:44.137878 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:44.138649 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:44.138656 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:44.138811 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:44.139461 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:44.139478 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:44.142126 (+ 2648us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":418,"threads_started":1}
W20260925 09:27:44.142565 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:44.133680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:38241 (local address 127.10.114.126:43579)
0925 09:27:44.134028 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:44.134036 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:44.134375 (+ 339us) server_negotiation.cc:407] Connection header received
0925 09:27:44.134617 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:44.134625 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:44.134724 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:44.134847 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:44.135912 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:44.136816 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:44.138007 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:44.138457 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:44.138583 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:44.142309 (+ 3726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.65:38241: BlockingRecv error: recv got EOF from 127.10.114.65:38241 (error 108)
Metrics: {"server-negotiator.queue_time_us":158}
Sep 25 09:27:44 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:44 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:44.584007 19968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:44.575429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.66:35745)
0925 09:27:44.575976 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:44.575993 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:44.576102 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:44.576420 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:44.576424 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:44.576810 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:44.576960 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:44.576968 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:44.578549 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:44.578554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:44.579544 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:44.579555 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:44.580432 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:44.580437 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:44.580546 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:44.581078 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:44.581095 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:44.583708 (+ 2613us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":266,"threads_started":1}
W20260925 09:27:44.584151 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:44.575485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:35745 (local address 127.10.114.126:43579)
0925 09:27:44.575730 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:44.575738 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:44.576079 (+ 341us) server_negotiation.cc:407] Connection header received
0925 09:27:44.576200 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:44.576203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:44.576275 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:44.576364 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:44.577103 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:44.578438 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:44.579673 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:44.580257 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:44.580392 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:44.583966 (+ 3574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.66:35745: BlockingRecv error: recv got EOF from 127.10.114.66:35745 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Sep 25 09:27:45 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:45 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:45.026979 19969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:45.018714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.67:43011)
0925 09:27:45.019265 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:45.019280 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:45.019450 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:45.019731 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:45.019734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:45.020158 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:45.020320 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:45.020329 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:45.021290 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:45.021293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:45.022317 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:45.022327 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:45.023109 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:45.023118 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:45.023288 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:45.024009 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:45.024026 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:45.026630 (+ 2604us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":302,"threads_started":1}
W20260925 09:27:45.027060 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:45.018844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:43011 (local address 127.10.114.126:43579)
0925 09:27:45.019017 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:45.019022 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:45.019418 (+ 396us) server_negotiation.cc:407] Connection header received
0925 09:27:45.019549 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:45.019553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:45.019612 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:45.019695 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:45.020453 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:45.021174 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:45.022448 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:45.022911 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:45.023050 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:45.026811 (+ 3761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.67:43011: BlockingRecv error: recv got EOF from 127.10.114.67:43011 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 25 09:27:45 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:45 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:45.152177 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:45.143434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.65:60581)
0925 09:27:45.143989 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:45.144005 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:45.144114 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:45.144407 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:45.144410 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:45.144777 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:45.144931 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:45.144939 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:45.145923 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:45.145927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:45.147084 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:45.147098 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:45.147833 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:45.147838 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:45.147969 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:45.148533 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:45.148551 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:45.151922 (+ 3371us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":312,"threads_started":1}
W20260925 09:27:45.152313 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:45.143615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:60581 (local address 127.10.114.126:43579)
0925 09:27:45.143897 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:45.143903 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:45.144077 (+ 174us) server_negotiation.cc:407] Connection header received
0925 09:27:45.144229 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:45.144233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:45.144292 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:45.144364 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:45.145043 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:45.145764 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:45.147256 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:45.147672 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:45.147765 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:45.152091 (+ 4326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.65:60581: BlockingRecv error: recv got EOF from 127.10.114.65:60581 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Sep 25 09:27:45 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:45 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:45.592641 19971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:45.585007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.66:45229)
0925 09:27:45.585495 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:45.585510 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:45.585611 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:45.585881 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:45.585884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:45.586235 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:45.586411 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:45.586419 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:45.587384 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:45.587388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:45.588337 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:45.588347 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:45.588966 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:45.588969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:45.589086 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:45.589625 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:45.589640 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:45.592323 (+ 2683us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":279,"threads_started":1}
W20260925 09:27:45.592679 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:45.585038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:45229 (local address 127.10.114.126:43579)
0925 09:27:45.585209 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:45.585213 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:45.585579 (+ 366us) server_negotiation.cc:407] Connection header received
0925 09:27:45.585702 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:45.585706 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:45.585764 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:45.585840 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:45.586530 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:45.587273 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:45.588458 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:45.588847 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:45.588940 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:45.592468 (+ 3528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.66:45229: BlockingRecv error: recv got EOF from 127.10.114.66:45229 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 25 09:27:46 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:46 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:46.039655 19972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:46.027980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.67:53343)
0925 09:27:46.028457 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:46.028472 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:46.028581 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:46.028864 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:46.028867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:46.029225 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:46.029381 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:46.029389 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:46.030363 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:46.030367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:46.031256 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:46.031266 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:46.032034 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:46.032038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:46.032154 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:46.036093 (+ 3939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:46.036125 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:46.039343 (+ 3218us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":259,"threads_started":1}
W20260925 09:27:46.039712 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:46.027961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:53343 (local address 127.10.114.126:43579)
0925 09:27:46.028163 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:46.028174 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:46.028552 (+ 378us) server_negotiation.cc:407] Connection header received
0925 09:27:46.028672 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:46.028675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:46.028743 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:46.028827 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:46.029509 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:46.030255 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:46.031389 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:46.031876 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:46.031972 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:46.039501 (+ 7529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.67:53343: BlockingRecv error: recv got EOF from 127.10.114.67:53343 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 25 09:27:46 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:46 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:46.161021 19973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:46.153116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.65:48205)
0925 09:27:46.153642 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:46.153681 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:46.153827 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:46.154210 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:46.154214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:46.154725 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:46.154882 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:46.154892 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:46.155953 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:46.155957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:46.156844 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:46.156853 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:46.157521 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:46.157525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:46.157631 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:46.158218 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:46.158235 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:46.160824 (+ 2589us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":282,"threads_started":1}
W20260925 09:27:46.161286 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:46.153182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:48205 (local address 127.10.114.126:43579)
0925 09:27:46.153399 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:46.153406 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:46.153793 (+ 387us) server_negotiation.cc:407] Connection header received
0925 09:27:46.153942 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:46.153947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:46.154026 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:46.154119 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:46.155037 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:46.155849 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:46.156956 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:46.157395 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:46.157487 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:46.161009 (+ 3522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.65:48205: BlockingRecv error: recv got EOF from 127.10.114.65:48205 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 25 09:27:46 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:46 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:46.602360 19974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:46.593575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.66:52647)
0925 09:27:46.594093 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:46.594109 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:46.594262 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:46.594610 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:46.594615 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:46.595018 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:46.595185 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:46.595194 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:46.596269 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:46.596275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:46.597216 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:46.597226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:46.597899 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:46.597904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:46.598062 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:46.598827 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:46.598852 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:46.602049 (+ 3197us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":288,"threads_started":1}
W20260925 09:27:46.602468 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:46.593642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:52647 (local address 127.10.114.126:43579)
0925 09:27:46.593915 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:46.593924 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:46.594203 (+ 279us) server_negotiation.cc:407] Connection header received
0925 09:27:46.594409 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:46.594414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:46.594481 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:46.594582 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:46.595352 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:46.596147 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:46.597354 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:46.597749 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:46.597872 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:46.602227 (+ 4355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.66:52647: BlockingRecv error: recv got EOF from 127.10.114.66:52647 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
Sep 25 09:27:47 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:47 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:47.048576 19975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:47.040691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.67:39349)
0925 09:27:47.041181 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:47.041202 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:47.041335 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:47.041677 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:47.041682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:47.042020 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:47.042154 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:47.042161 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:47.043072 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:47.043075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:47.044044 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:47.044054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:47.044675 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:47.044679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:47.044780 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:47.045382 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:47.045398 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:47.048205 (+ 2807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":265,"threads_started":1}
W20260925 09:27:47.048611 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:47.040824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:39349 (local address 127.10.114.126:43579)
0925 09:27:47.041034 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:47.041041 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:47.041271 (+ 230us) server_negotiation.cc:407] Connection header received
0925 09:27:47.041438 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:47.041442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:47.041536 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:47.041634 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:47.042273 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:47.042959 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:47.044181 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:47.044554 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:47.044650 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:47.048382 (+ 3732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.67:39349: BlockingRecv error: recv got EOF from 127.10.114.67:39349 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 25 09:27:47 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:47 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:47.169956 19976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:47.161851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.65:53467)
0925 09:27:47.162338 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:47.162354 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:47.162462 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:47.162759 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:47.162762 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:47.163178 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:47.163347 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:47.163354 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:47.164264 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:47.164268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:47.165135 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:47.165145 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:47.165788 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:47.165792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:47.165925 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:47.166479 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:47.166495 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:47.169616 (+ 3121us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":271,"threads_started":1}
W20260925 09:27:47.170111 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:47.161967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:53467 (local address 127.10.114.126:43579)
0925 09:27:47.162137 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:47.162141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:47.162435 (+ 294us) server_negotiation.cc:407] Connection header received
0925 09:27:47.162560 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:47.162563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:47.162629 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:47.162710 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:47.163458 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:47.164158 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:47.165250 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:47.165671 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:47.165795 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:47.169854 (+ 4059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.65:53467: BlockingRecv error: recv got EOF from 127.10.114.65:53467 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 25 09:27:47 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:47 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:47.611088 19977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:47.603346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.66:55005)
0925 09:27:47.603873 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:47.603889 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:47.603995 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:47.604298 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:47.604301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:47.604667 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:47.604840 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:47.604848 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:47.605759 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:47.605762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:47.606629 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:47.606657 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:47.607317 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:47.607322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:47.607440 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:47.608070 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:47.608088 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:47.610845 (+ 2757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":269,"threads_started":1}
W20260925 09:27:47.611276 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:47.603413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:55005 (local address 127.10.114.126:43579)
0925 09:27:47.603560 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:47.603564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:47.603956 (+ 392us) server_negotiation.cc:407] Connection header received
0925 09:27:47.604085 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:47.604089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:47.604144 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:47.604227 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:47.604957 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:47.605642 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:47.606765 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:47.607174 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:47.607261 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:47.611076 (+ 3815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.66:55005: BlockingRecv error: recv got EOF from 127.10.114.66:55005 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 25 09:27:48 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:48 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:48.057466 19978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:48.049540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.67:55667)
0925 09:27:48.050015 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:48.050034 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:48.050186 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:48.050515 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:48.050519 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:48.050917 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:48.051092 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:48.051100 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:48.052230 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:48.052238 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:48.053095 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:48.053105 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:48.053773 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:48.053779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:48.053948 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:48.054542 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:48.054557 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:48.057106 (+ 2549us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":273,"threads_started":1}
W20260925 09:27:48.057548 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:48.049595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:55667 (local address 127.10.114.126:43579)
0925 09:27:48.049792 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:48.049798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:48.050120 (+ 322us) server_negotiation.cc:407] Connection header received
0925 09:27:48.050333 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:48.050337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:48.050394 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:48.050491 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:48.051249 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:48.052114 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:48.053227 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:48.053624 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:48.053743 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:48.057308 (+ 3565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.67:55667: BlockingRecv error: recv got EOF from 127.10.114.67:55667 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 25 09:27:48 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:48 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:48.179553 19979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:48.170942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.65:52645)
0925 09:27:48.171470 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:48.171487 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:48.171603 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:48.171972 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:48.171975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:48.172388 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:48.172551 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:48.172560 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:48.173490 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:48.173494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:48.174350 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:48.174361 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:48.175021 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:48.175025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:48.175156 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:48.175838 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:48.175857 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:48.179167 (+ 3310us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":301,"threads_started":1}
W20260925 09:27:48.179668 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:48.171018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:52645 (local address 127.10.114.126:43579)
0925 09:27:48.171171 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:48.171175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:48.171562 (+ 387us) server_negotiation.cc:407] Connection header received
0925 09:27:48.171711 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:48.171715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:48.171840 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:48.171918 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:48.172674 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:48.173373 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:48.174468 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:48.174901 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:48.174988 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:48.179399 (+ 4411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.65:52645: BlockingRecv error: recv got EOF from 127.10.114.65:52645 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 25 09:27:48 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:48 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:48.621776 19980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:48.612148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.66:59819)
0925 09:27:48.612974 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:48.612999 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:48.613175 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:48.613603 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:48.613610 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:48.614042 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:48.614218 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:48.614227 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:48.615213 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:48.615218 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:48.616329 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:48.616341 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:48.617391 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:48.617398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:48.617626 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:48.618408 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:48.618429 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:48.621561 (+ 3132us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":524,"threads_started":1}
W20260925 09:27:48.621976 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:48.612423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:59819 (local address 127.10.114.126:43579)
0925 09:27:48.612621 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:48.612630 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:48.613112 (+ 482us) server_negotiation.cc:407] Connection header received
0925 09:27:48.613320 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:48.613325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:48.613424 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:48.613535 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:48.614382 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:48.615079 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:48.616529 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:48.617193 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:48.617384 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:48.621742 (+ 4358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.66:59819: BlockingRecv error: recv got EOF from 127.10.114.66:59819 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 25 09:27:49 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:49 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:49.066010 19981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:49.058404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.67:43237)
0925 09:27:49.058837 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:49.058850 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:49.058993 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:49.059326 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:49.059331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:49.059704 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:49.059916 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:49.059924 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:49.060920 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:49.060926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:49.061829 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:49.061839 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:49.062495 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:49.062500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:49.062662 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:49.063229 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:49.063244 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:49.065700 (+ 2456us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":250,"threads_started":1}
W20260925 09:27:49.066094 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:49.058460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:43237 (local address 127.10.114.126:43579)
0925 09:27:49.058644 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:49.058652 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:49.058932 (+ 280us) server_negotiation.cc:407] Connection header received
0925 09:27:49.059141 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:49.059144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:49.059199 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:49.059289 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:49.060063 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:49.060780 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:49.061984 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:49.062369 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:49.062487 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:49.065874 (+ 3387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.67:43237: BlockingRecv error: recv got EOF from 127.10.114.67:43237 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 25 09:27:49 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:49 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:49.189347 19982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:49.180700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.65:54277)
0925 09:27:49.181213 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:49.181228 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:49.181341 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:49.181650 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:49.181652 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:49.181997 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:49.182223 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:49.182232 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:49.183230 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:49.183233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:49.184547 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:49.184563 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:49.185550 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:49.185554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:49.185666 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:49.186267 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:49.186283 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:49.189118 (+ 2835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":267,"threads_started":1}
W20260925 09:27:49.189546 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:49.180776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:54277 (local address 127.10.114.126:43579)
0925 09:27:49.181020 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:49.181027 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:49.181308 (+ 281us) server_negotiation.cc:407] Connection header received
0925 09:27:49.181455 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:49.181459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:49.181530 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:49.181613 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:49.182353 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:49.183093 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:49.184714 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:49.185377 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:49.185470 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:49.189284 (+ 3814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.65:54277: BlockingRecv error: recv got EOF from 127.10.114.65:54277 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 25 09:27:49 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:49 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:49.631009 19983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:49.622579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.66:40599)
0925 09:27:49.623103 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:49.623117 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:49.623274 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:49.623607 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:49.623611 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:49.624032 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:49.624187 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:49.624194 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:49.625201 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:49.625205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:49.626603 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:49.626621 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:49.627358 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:49.627363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:49.627472 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:49.628140 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:49.628157 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:49.630785 (+ 2628us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":265,"threads_started":1}
W20260925 09:27:49.631188 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:49.622751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:40599 (local address 127.10.114.126:43579)
0925 09:27:49.622983 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:49.622990 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:49.623206 (+ 216us) server_negotiation.cc:407] Connection header received
0925 09:27:49.623398 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:49.623403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:49.623483 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:49.623583 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:49.624316 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:49.625034 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:49.626733 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:49.627199 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:49.627291 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:49.630957 (+ 3666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.66:40599: BlockingRecv error: recv got EOF from 127.10.114.66:40599 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Sep 25 09:27:50 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:50 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:50.075676 19984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:50.067010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.67:44549)
0925 09:27:50.067520 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:50.067535 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:50.067642 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:50.067963 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:50.067966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:50.068316 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:50.068479 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:50.068486 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:50.069850 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:50.069855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:50.071462 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:50.071474 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:50.072141 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:50.072145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:50.072253 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:50.072810 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:50.072825 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:50.075462 (+ 2637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":265,"threads_started":1}
W20260925 09:27:50.075853 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:50.067081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:44549 (local address 127.10.114.126:43579)
0925 09:27:50.067270 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:50.067281 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:50.067631 (+ 350us) server_negotiation.cc:407] Connection header received
0925 09:27:50.067739 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:50.067742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:50.067823 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:50.067923 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:50.068623 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:50.069696 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:50.071590 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:50.071998 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:50.072087 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:50.075629 (+ 3542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.67:44549: BlockingRecv error: recv got EOF from 127.10.114.67:44549 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 25 09:27:50 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:50 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:50.198454 19985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:50.190269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.65:43459)
0925 09:27:50.190767 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:50.190782 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:50.190947 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:50.191299 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:50.191306 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:50.191686 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:50.191891 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:50.191901 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:50.192952 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:50.192960 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:50.193864 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:50.193875 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:50.194576 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:50.194582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:50.194747 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:50.195315 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:50.195333 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:50.198135 (+ 2802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":280,"threads_started":1}
W20260925 09:27:50.198685 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:50.190359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:43459 (local address 127.10.114.126:43579)
0925 09:27:50.190674 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:50.190688 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:50.190880 (+ 192us) server_negotiation.cc:407] Connection header received
0925 09:27:50.191079 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:50.191084 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:50.191159 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:50.191264 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:50.192036 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:50.192810 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:50.194008 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:50.194431 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:50.194555 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:50.198412 (+ 3857us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.65:43459: BlockingRecv error: recv got EOF from 127.10.114.65:43459 (error 108)
Metrics: {"server-negotiator.queue_time_us":190}
Sep 25 09:27:50 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:50 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:50.640345 19986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:50.631866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.66:46537)
0925 09:27:50.632518 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:50.632543 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:50.632656 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:50.632917 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:50.632920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:50.633284 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:50.633451 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:50.633458 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:50.634521 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:50.634525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:50.635450 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:50.635461 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:50.636234 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:50.636239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:50.636458 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:50.637048 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:50.637066 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:50.640079 (+ 3013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":476,"threads_started":1}
W20260925 09:27:50.640457 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:50.631929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:46537 (local address 127.10.114.126:43579)
0925 09:27:50.632162 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:50.632172 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:50.632616 (+ 444us) server_negotiation.cc:407] Connection header received
0925 09:27:50.632743 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:50.632747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:50.632802 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:50.632885 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:50.633587 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:50.634383 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:50.635601 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:50.636102 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:50.636198 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:50.640255 (+ 4057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.66:46537: BlockingRecv error: recv got EOF from 127.10.114.66:46537 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 25 09:27:51 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:51 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.67@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:51.091611 19987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:51.076693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.67:54529)
0925 09:27:51.077196 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:51.077211 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:51.077371 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:51.077698 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:51.077703 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:51.078070 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:51.078254 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:51.078263 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:51.079345 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:51.079353 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:51.080509 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:51.080533 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:51.081333 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:51.081338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:51.081463 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:51.088328 (+ 6865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:51.088346 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:51.091308 (+ 2962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":256,"threads_started":1}
W20260925 09:27:51.091859 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:51.076762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:54529 (local address 127.10.114.126:43579)
0925 09:27:51.077031 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:51.077037 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:51.077295 (+ 258us) server_negotiation.cc:407] Connection header received
0925 09:27:51.077502 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:51.077509 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:51.077571 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:51.077670 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:51.078396 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:51.079192 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:51.080672 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:51.081183 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:51.081296 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:51.091568 (+ 10272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.67:54529: BlockingRecv error: recv got EOF from 127.10.114.67:54529 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 25 09:27:51 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:51 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.65@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:51.207868 19988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:51.199639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.65:59709)
0925 09:27:51.200158 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:51.200175 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:51.200315 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:51.200679 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:51.200683 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:51.201118 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:51.201272 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:51.201280 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:51.202265 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:51.202272 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:51.203156 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:51.203166 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:51.203963 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:51.203968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:51.204138 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:51.204705 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:51.204721 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:51.207491 (+ 2770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":293,"threads_started":1}
W20260925 09:27:51.208109 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:51.199791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:59709 (local address 127.10.114.126:43579)
0925 09:27:51.200013 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:51.200020 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:51.200259 (+ 239us) server_negotiation.cc:407] Connection header received
0925 09:27:51.200478 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:51.200483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:51.200554 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:51.200653 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:51.201414 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:51.202136 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:51.203289 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:51.203760 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:51.203956 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:51.207855 (+ 3899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.65:59709: BlockingRecv error: recv got EOF from 127.10.114.65:59709 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 25 09:27:51 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:27:51 dist-test-slave-9pj8 krb5kdc[18944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.114.66@KRBTEST.COM for oryx/127.10.114.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:27:51.649699 19989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:51.641554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:43579 (local address 127.10.114.66:34819)
0925 09:27:51.642032 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:51.642047 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:51.642190 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:51.642521 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:51.642525 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:51.642943 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:51.643107 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:51.643116 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:51.644191 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:51.644199 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:51.645096 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:51.645107 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:51.645811 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:51.645816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:51.645986 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:51.646616 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:51.646646 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:51.649365 (+ 2719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.114.126:43579: Server oryx/127.10.114.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":286,"threads_started":1}
W20260925 09:27:51.649739 19680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:51.641640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:34819 (local address 127.10.114.126:43579)
0925 09:27:51.641822 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:51.641830 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:51.642127 (+ 297us) server_negotiation.cc:407] Connection header received
0925 09:27:51.642317 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:51.642320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:51.642398 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:51.642492 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:51.643245 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:51.644021 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:51.645226 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:51.645666 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:51.645812 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:51.649526 (+ 3714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.114.66:34819: BlockingRecv error: recv got EOF from 127.10.114.66:34819 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 09:27:51.927225 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 19531
I20260925 09:27:51.939711 19991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:51.928337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39752 (local address 127.10.114.65:38345)
0925 09:27:51.928861 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:51.928869 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:51.928895 (+ 26us) server_negotiation.cc:407] Connection header received
0925 09:27:51.928948 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:51.928954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:51.929226 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:51.929446 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:51.931083 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:51.932426 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:51.933585 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:51.934116 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:51.934226 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:51.936710 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:51.936719 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:51.936733 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:51.936766 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:51.939225 (+ 2459us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.114.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0925 09:27:51.939459 (+ 234us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39752: Request ticket server kudu/127.10.114.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":268,"threads_started":1}
W20260925 09:27:51.939836 19991 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39752: Request ticket server kudu/127.10.114.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260925 09:27:51.939888 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 19531: Not authorized: Client connection negotiation failed: client connection to 127.10.114.65:38345: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.114.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260925 09:27:51.940255 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 19531
I20260925 09:27:51.956486 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 19682
I20260925 09:27:51.968528 19992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:51.957652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57080 (local address 127.10.114.66:42377)
0925 09:27:51.958263 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:51.958270 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:51.958298 (+ 28us) server_negotiation.cc:407] Connection header received
0925 09:27:51.958351 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:51.958359 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:51.958576 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:51.958728 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:51.960287 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:51.961322 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:51.962614 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:51.963150 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:51.963238 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:51.965574 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:51.965583 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:51.965597 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:51.965634 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:51.968067 (+ 2433us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.114.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0925 09:27:51.968298 (+ 231us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57080: Request ticket server kudu/127.10.114.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":308,"threads_started":1}
W20260925 09:27:51.968639 19992 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57080: Request ticket server kudu/127.10.114.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260925 09:27:51.968775 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 19682: Not authorized: Client connection negotiation failed: client connection to 127.10.114.66:42377: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.114.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260925 09:27:51.968942 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 19682
I20260925 09:27:51.984819 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 19813
I20260925 09:27:51.995775 19993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:51.985683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56190 (local address 127.10.114.67:44669)
0925 09:27:51.986195 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:51.986202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:51.986225 (+ 23us) server_negotiation.cc:407] Connection header received
0925 09:27:51.986282 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:51.986288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:51.986523 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:51.986660 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:51.987925 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:51.988931 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:51.990038 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:51.990583 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:51.990676 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:51.992979 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:51.992987 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:51.992999 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:51.993027 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:51.995278 (+ 2251us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.114.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0925 09:27:51.995504 (+ 226us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56190: Request ticket server kudu/127.10.114.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":263,"threads_started":1}
W20260925 09:27:51.995882 19993 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56190: Request ticket server kudu/127.10.114.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260925 09:27:51.995867 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 19813: Not authorized: Client connection negotiation failed: client connection to 127.10.114.67:44669: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.114.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260925 09:27:51.996018 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 19813
I20260925 09:27:52.012022 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 19462
I20260925 09:27:52.013262 19524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:52.122507 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 19462
2026-09-25T09:27:52Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (20854 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-25T09:27:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:27:52Z Disabled control of system clock
I20260925 09:27:52.193427 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:42785
--webserver_interface=127.10.114.126
--webserver_port=0
--builtin_ntp_servers=127.10.114.84:38375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:42785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 09:27:52.499955 19998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:52.500273 19998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:52.500432 19998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:52.508774 19998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:27:52.508894 19998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:52.508980 19998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:52.509065 19998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:27:52.509128 19998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:27:52.518059 19998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:42785
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:42785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:52.519441 19998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:52.521345 19998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:52.533314 20004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:52.533372 20006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:52.533347 20003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:52.537132 19998 server_base.cc:1034] running on GCE node
I20260925 09:27:52.537916 19998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:52.539351 19998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:52.540568 19998 hybrid_clock.cc:648] HybridClock initialized: now 1790328472540533 us; error 37 us; skew 500 ppm
I20260925 09:27:52.540910 19998 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:27:52.543061 19998 webserver.cc:460] Webserver started at http://127.10.114.126:37523/ using document root <none> and password file <none>
I20260925 09:27:52.543732 19998 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:52.543892 19998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:52.544214 19998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:52.546973 19998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "de34dd15198c489c871c474ac5ad6e8a"
format_stamp: "Formatted at 2026-09-25 09:27:52 on dist-test-slave-9pj8"
server_key: "4449e33c3b766742997c58abc2a52ad2"
server_key_iv: "934837252330ef29ddb94a42b8263ce4"
server_key_version: "encryptionkey@0"
I20260925 09:27:52.547879 19998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "de34dd15198c489c871c474ac5ad6e8a"
format_stamp: "Formatted at 2026-09-25 09:27:52 on dist-test-slave-9pj8"
server_key: "4449e33c3b766742997c58abc2a52ad2"
server_key_iv: "934837252330ef29ddb94a42b8263ce4"
server_key_version: "encryptionkey@0"
I20260925 09:27:52.552738 19998 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260925 09:27:52.555960 20012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:52.557188 19998 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:27:52.557358 19998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "de34dd15198c489c871c474ac5ad6e8a"
format_stamp: "Formatted at 2026-09-25 09:27:52 on dist-test-slave-9pj8"
server_key: "4449e33c3b766742997c58abc2a52ad2"
server_key_iv: "934837252330ef29ddb94a42b8263ce4"
server_key_version: "encryptionkey@0"
I20260925 09:27:52.557562 19998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:52.588728 19998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:52.589547 19998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:52.589834 19998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:52.608260 19998 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:42785
I20260925 09:27:52.608285 20063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:42785 every 8 connection(s)
I20260925 09:27:52.609603 19998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:27:52.610664 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 19998
I20260925 09:27:52.610900 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:27:52.611841 10697 external_mini_cluster.cc:1428] Setting key 6e63c916115c4d68b3567281e88f00f8
I20260925 09:27:52.614777 20064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:52.625571 20067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:52.614044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57542 (local address 127.10.114.126:42785)
0925 09:27:52.615034 (+ 990us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:52.615045 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:52.615089 (+ 44us) server_negotiation.cc:407] Connection header received
0925 09:27:52.616735 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:52.616756 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:52.617212 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:52.617628 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:52.618205 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:52.619555 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:52.620986 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:52.621650 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:52.621850 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:52.623075 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:52.623082 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:52.623097 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:52.623698 (+ 601us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:52.623818 (+ 120us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:52.623847 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:52.623967 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:52.624241 (+ 274us) server_negotiation.cc:299] Negotiation successful
0925 09:27:52.624605 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":318,"threads_started":1}
I20260925 09:27:52.627004 20064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a: Bootstrap starting.
I20260925 09:27:52.630198 20064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:52.631259 20064 log.cc:824] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:52.634136 20064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a: No bootstrap required, opened a new log
I20260925 09:27:52.641907 20064 raft_consensus.cc:348] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de34dd15198c489c871c474ac5ad6e8a" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42785 } }
I20260925 09:27:52.642320 20064 raft_consensus.cc:374] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:52.642439 20064 raft_consensus.cc:729] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de34dd15198c489c871c474ac5ad6e8a, State: Initialized, Role: FOLLOWER
I20260925 09:27:52.643042 20064 consensus_queue.cc:260] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de34dd15198c489c871c474ac5ad6e8a" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42785 } }
I20260925 09:27:52.643301 20064 raft_consensus.cc:388] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:27:52.643460 20064 raft_consensus.cc:482] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:27:52.643646 20064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:52.645951 20064 raft_consensus.cc:504] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de34dd15198c489c871c474ac5ad6e8a" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42785 } }
I20260925 09:27:52.646596 20064 leader_election.cc:302] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de34dd15198c489c871c474ac5ad6e8a; no voters:
I20260925 09:27:52.647199 20064 leader_election.cc:288] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:27:52.647343 20069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:52.648175 20069 raft_consensus.cc:686] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a [term 1 LEADER]: Becoming Leader. State: Replica: de34dd15198c489c871c474ac5ad6e8a, State: Running, Role: LEADER
I20260925 09:27:52.648654 20069 consensus_queue.cc:237] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de34dd15198c489c871c474ac5ad6e8a" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42785 } }
I20260925 09:27:52.649544 20064 sys_catalog.cc:564] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:27:52.651557 20071 sys_catalog.cc:455] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader de34dd15198c489c871c474ac5ad6e8a. Latest consensus state: current_term: 1 leader_uuid: "de34dd15198c489c871c474ac5ad6e8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de34dd15198c489c871c474ac5ad6e8a" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42785 } } }
I20260925 09:27:52.652086 20071 sys_catalog.cc:458] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:52.652422 20070 sys_catalog.cc:455] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de34dd15198c489c871c474ac5ad6e8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de34dd15198c489c871c474ac5ad6e8a" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42785 } } }
I20260925 09:27:52.652838 20070 sys_catalog.cc:458] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:52.654178 20078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:27:52.659420 20078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:27:52.668166 20078 catalog_manager.cc:1347] Generated new cluster ID: e14fc57ec49b43a7b64b9010fe74d741
I20260925 09:27:52.668267 20078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:27:52.694094 20078 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:27:52.695451 20078 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:27:52.704634 20078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de34dd15198c489c871c474ac5ad6e8a: Generated new TSK 0
I20260925 09:27:52.705514 20078 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:27:52.709695 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42785
--builtin_ntp_servers=127.10.114.84:38375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 09:27:53.048097 20088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:53.048350 20088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:53.048430 20088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:53.055294 20088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:53.055380 20088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:53.055544 20088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:27:53.064453 20088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:53.065661 20088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:53.067325 20088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:53.080125 20096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:53.080078 20093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:53.080024 20094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:53.080914 20088 server_base.cc:1034] running on GCE node
I20260925 09:27:53.081476 20088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:53.082355 20088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:53.083570 20088 hybrid_clock.cc:648] HybridClock initialized: now 1790328473083507 us; error 53 us; skew 500 ppm
I20260925 09:27:53.083922 20088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:27:53.086021 20088 webserver.cc:460] Webserver started at http://127.10.114.65:39235/ using document root <none> and password file <none>
I20260925 09:27:53.086659 20088 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:53.086767 20088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:53.087108 20088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:53.089908 20088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "b7aad9c2db5c4fc2817356b7de18e04d"
format_stamp: "Formatted at 2026-09-25 09:27:53 on dist-test-slave-9pj8"
server_key: "d58d270dd165f856b2c4a139589ba231"
server_key_iv: "65d4b70bf251a3d786112b973b7d1623"
server_key_version: "encryptionkey@0"
I20260925 09:27:53.090668 20088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "b7aad9c2db5c4fc2817356b7de18e04d"
format_stamp: "Formatted at 2026-09-25 09:27:53 on dist-test-slave-9pj8"
server_key: "d58d270dd165f856b2c4a139589ba231"
server_key_iv: "65d4b70bf251a3d786112b973b7d1623"
server_key_version: "encryptionkey@0"
I20260925 09:27:53.095520 20088 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260925 09:27:53.098757 20102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:53.100188 20088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:27:53.100376 20088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "b7aad9c2db5c4fc2817356b7de18e04d"
format_stamp: "Formatted at 2026-09-25 09:27:53 on dist-test-slave-9pj8"
server_key: "d58d270dd165f856b2c4a139589ba231"
server_key_iv: "65d4b70bf251a3d786112b973b7d1623"
server_key_version: "encryptionkey@0"
I20260925 09:27:53.100605 20088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:53.118808 20088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:53.119673 20088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:53.120080 20088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:53.121016 20088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:53.122607 20088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:53.122738 20088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:53.122891 20088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:53.122985 20088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:53.156987 20088 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:34919
I20260925 09:27:53.157073 20214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:34919 every 8 connection(s)
I20260925 09:27:53.158277 20088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:27:53.164810 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 20088
I20260925 09:27:53.165032 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:27:53.165725 10697 external_mini_cluster.cc:1428] Setting key ffa70d27fb4fd27c98ee8b1372b1881b
I20260925 09:27:53.168385 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42785
--builtin_ntp_servers=127.10.114.84:38375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260925 09:27:53.171271 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:53.161105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:58691 (local address 127.10.114.126:42785)
0925 09:27:53.161633 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:53.161638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:53.162865 (+ 1227us) server_negotiation.cc:407] Connection header received
0925 09:27:53.164621 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:53.164626 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:53.164714 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:53.164831 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:53.165962 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:53.166901 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:53.168653 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:53.169041 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:53.169173 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:53.170480 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:53.170490 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:53.170494 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:53.170509 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:53.170545 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:53.170556 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:53.170648 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:53.170869 (+ 221us) server_negotiation.cc:299] Negotiation successful
0925 09:27:53.170960 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":299,"threads_started":1}
I20260925 09:27:53.172288 20218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:53.161764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42785 (local address 127.10.114.65:58691)
0925 09:27:53.162713 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:53.162753 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:53.164368 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:53.165065 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:53.165074 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:53.165201 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:53.165758 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:53.165774 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:53.167047 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:53.167051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:53.168464 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:53.168488 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:53.169248 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:53.169256 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:53.169482 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:53.170176 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:27:53.170179 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:53.170216 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:27:53.170286 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:53.170658 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:53.170670 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:53.170684 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 09:27:53.170859 (+ 175us) client_negotiation.cc:240] Negotiation successful
0925 09:27:53.171235 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":276,"threads_started":1}
I20260925 09:27:53.173602 20215 heartbeater.cc:347] Connected to a master server at 127.10.114.126:42785
I20260925 09:27:53.173892 20215 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:53.174624 20215 heartbeater.cc:511] Master 127.10.114.126:42785 requested a full tablet report, sending...
I20260925 09:27:53.176690 20029 ts_manager.cc:194] Registered new tserver with Master: b7aad9c2db5c4fc2817356b7de18e04d (127.10.114.65:34919)
I20260925 09:27:53.178138 20029 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.114.65:58691
W20260925 09:27:53.493852 20219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:53.494163 20219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:53.494325 20219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:53.501564 20219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:53.501698 20219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:53.501868 20219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:27:53.510788 20219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:53.512058 20219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:53.513777 20219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:53.526489 20227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:53.526513 20224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:53.526547 20225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:53.527684 20219 server_base.cc:1034] running on GCE node
I20260925 09:27:53.528272 20219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:53.529194 20219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:53.530431 20219 hybrid_clock.cc:648] HybridClock initialized: now 1790328473530389 us; error 62 us; skew 500 ppm
I20260925 09:27:53.530745 20219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:27:53.533010 20219 webserver.cc:460] Webserver started at http://127.10.114.66:42745/ using document root <none> and password file <none>
I20260925 09:27:53.533712 20219 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:53.533841 20219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:53.534159 20219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:53.536893 20219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "0831e461dd9a4feabc4b3a428aba95da"
format_stamp: "Formatted at 2026-09-25 09:27:53 on dist-test-slave-9pj8"
server_key: "e32a41bc6759f019763c9fbc0b3e014b"
server_key_iv: "e162d10da919b93083a8dea2fd507576"
server_key_version: "encryptionkey@0"
I20260925 09:27:53.537711 20219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "0831e461dd9a4feabc4b3a428aba95da"
format_stamp: "Formatted at 2026-09-25 09:27:53 on dist-test-slave-9pj8"
server_key: "e32a41bc6759f019763c9fbc0b3e014b"
server_key_iv: "e162d10da919b93083a8dea2fd507576"
server_key_version: "encryptionkey@0"
I20260925 09:27:53.542730 20219 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260925 09:27:53.546168 20233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:53.547431 20219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 09:27:53.547616 20219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "0831e461dd9a4feabc4b3a428aba95da"
format_stamp: "Formatted at 2026-09-25 09:27:53 on dist-test-slave-9pj8"
server_key: "e32a41bc6759f019763c9fbc0b3e014b"
server_key_iv: "e162d10da919b93083a8dea2fd507576"
server_key_version: "encryptionkey@0"
I20260925 09:27:53.547843 20219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:53.569100 20219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:53.569824 20219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:53.570128 20219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:53.570990 20219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:53.572412 20219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:53.572541 20219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:53.572662 20219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:53.572746 20219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:53.604650 20219 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:35225
I20260925 09:27:53.604732 20345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:35225 every 8 connection(s)
I20260925 09:27:53.605995 20219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:27:53.610872 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 20219
I20260925 09:27:53.611047 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:27:53.611600 10697 external_mini_cluster.cc:1428] Setting key c9006b964d73da335c16b59621142b61
I20260925 09:27:53.614425 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42785
--builtin_ntp_servers=127.10.114.84:38375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260925 09:27:53.621287 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:53.608605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:45325 (local address 127.10.114.126:42785)
0925 09:27:53.608802 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:53.608811 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:53.610340 (+ 1529us) server_negotiation.cc:407] Connection header received
0925 09:27:53.612645 (+ 2305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:53.612651 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:53.612729 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:53.612814 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:53.614547 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:53.615684 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:53.618077 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:53.618704 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:53.618831 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:53.620464 (+ 1633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:53.620471 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:53.620476 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:53.620497 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:53.620534 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:53.620549 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:53.620636 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:53.620886 (+ 250us) server_negotiation.cc:299] Negotiation successful
0925 09:27:53.620976 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 09:27:53.622296 20348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:53.609141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42785 (local address 127.10.114.66:45325)
0925 09:27:53.610156 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:53.610202 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:53.612341 (+ 2139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:53.613469 (+ 1128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:53.613479 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:53.613586 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:53.614364 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:53.614389 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:53.615877 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:53.615893 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:53.617927 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:53.617944 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:53.618946 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:53.618955 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:53.619294 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:53.620110 (+ 816us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:27:53.620115 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:53.620166 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:27:53.620266 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:53.620653 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:53.620665 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:53.620675 (+ 10us) client_negotiation.cc:769] Sending connection context
0925 09:27:53.620838 (+ 163us) client_negotiation.cc:240] Negotiation successful
0925 09:27:53.621240 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":333,"threads_started":1}
I20260925 09:27:53.623670 20346 heartbeater.cc:347] Connected to a master server at 127.10.114.126:42785
I20260925 09:27:53.623987 20346 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:53.624694 20346 heartbeater.cc:511] Master 127.10.114.126:42785 requested a full tablet report, sending...
I20260925 09:27:53.625959 20029 ts_manager.cc:194] Registered new tserver with Master: 0831e461dd9a4feabc4b3a428aba95da (127.10.114.66:35225)
I20260925 09:27:53.626957 20029 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.114.66:45325
W20260925 09:27:53.926935 20349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:53.927174 20349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:53.927313 20349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:53.934566 20349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:53.934721 20349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:53.934877 20349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:27:53.944346 20349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:53.945438 20349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:53.947016 20349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:53.959537 20354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:53.959445 20357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:53.959478 20355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:53.960704 20349 server_base.cc:1034] running on GCE node
I20260925 09:27:53.961309 20349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:53.962201 20349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:53.963425 20349 hybrid_clock.cc:648] HybridClock initialized: now 1790328473963378 us; error 44 us; skew 500 ppm
I20260925 09:27:53.963711 20349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:27:53.965830 20349 webserver.cc:460] Webserver started at http://127.10.114.67:38533/ using document root <none> and password file <none>
I20260925 09:27:53.966471 20349 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:53.966614 20349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:53.966924 20349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:53.969542 20349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "c531aa42dcce465dbcc9956030dcf5a4"
format_stamp: "Formatted at 2026-09-25 09:27:53 on dist-test-slave-9pj8"
server_key: "a4058e774d4344759febce654b44094e"
server_key_iv: "3e800e54d27b11791deb7d4891f210b1"
server_key_version: "encryptionkey@0"
I20260925 09:27:53.970289 20349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "c531aa42dcce465dbcc9956030dcf5a4"
format_stamp: "Formatted at 2026-09-25 09:27:53 on dist-test-slave-9pj8"
server_key: "a4058e774d4344759febce654b44094e"
server_key_iv: "3e800e54d27b11791deb7d4891f210b1"
server_key_version: "encryptionkey@0"
I20260925 09:27:53.974938 20349 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260925 09:27:53.978125 20363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:53.979311 20349 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 09:27:53.979473 20349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "c531aa42dcce465dbcc9956030dcf5a4"
format_stamp: "Formatted at 2026-09-25 09:27:53 on dist-test-slave-9pj8"
server_key: "a4058e774d4344759febce654b44094e"
server_key_iv: "3e800e54d27b11791deb7d4891f210b1"
server_key_version: "encryptionkey@0"
I20260925 09:27:53.979666 20349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:53.994240 20349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:53.994937 20349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:53.995219 20349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:53.996096 20349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:53.997375 20349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:53.997489 20349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:53.997627 20349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:53.997711 20349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:54.030369 20349 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:41945
I20260925 09:27:54.030422 20475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:41945 every 8 connection(s)
I20260925 09:27:54.031617 20349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:27:54.037611 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 20349
I20260925 09:27:54.037786 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:27:54.038221 10697 external_mini_cluster.cc:1428] Setting key 8e2fa45d67696e5fb5c1e44f616e2364
I20260925 09:27:54.043802 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:54.034134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:53317 (local address 127.10.114.126:42785)
0925 09:27:54.034299 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:54.034304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:54.035623 (+ 1319us) server_negotiation.cc:407] Connection header received
0925 09:27:54.037332 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:54.037338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:54.037419 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:54.037534 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:54.038647 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:54.039415 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:54.041051 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:54.041458 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:54.041600 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:54.042968 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:54.042987 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:54.042994 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:54.043014 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:54.043041 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:54.043052 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:54.043163 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:54.043377 (+ 214us) server_negotiation.cc:299] Negotiation successful
0925 09:27:54.043474 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:27:54.044700 20478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:54.034658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42785 (local address 127.10.114.67:53317)
0925 09:27:54.035475 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:54.035510 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:54.037060 (+ 1550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:54.037767 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:54.037774 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:54.037891 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:54.038493 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:54.038511 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:54.039577 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:54.039583 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:54.040900 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:54.040911 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:54.041676 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:54.041682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:54.041920 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:54.042644 (+ 724us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:27:54.042646 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:54.042681 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:27:54.042760 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:54.043162 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:54.043173 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:54.043184 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 09:27:54.043363 (+ 179us) client_negotiation.cc:240] Negotiation successful
0925 09:27:54.043712 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":250,"threads_started":1}
I20260925 09:27:54.046025 20476 heartbeater.cc:347] Connected to a master server at 127.10.114.126:42785
I20260925 09:27:54.046270 20476 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:54.046931 20476 heartbeater.cc:511] Master 127.10.114.126:42785 requested a full tablet report, sending...
I20260925 09:27:54.048161 20029 ts_manager.cc:194] Registered new tserver with Master: c531aa42dcce465dbcc9956030dcf5a4 (127.10.114.67:41945)
I20260925 09:27:54.048924 20029 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.114.67:53317
I20260925 09:27:54.053366 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 09:27:54.059478 20217 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:54.057883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33158 (local address 127.10.114.126:42785)
0925 09:27:54.058064 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:54.058073 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:54.058600 (+ 527us) server_negotiation.cc:407] Connection header received
0925 09:27:54.058715 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:54.058718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:54.058769 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:54.058839 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:54.059301 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33158: BlockingRecv error: recv got EOF from 127.0.0.1:33158 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:27:54.059947 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 20088
I20260925 09:27:54.067451 20485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:54.060914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60734 (local address 127.10.114.65:34919)
0925 09:27:54.061469 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:54.061483 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:54.061508 (+ 25us) server_negotiation.cc:407] Connection header received
0925 09:27:54.061559 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:54.061563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:54.061727 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:54.061843 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:54.062182 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:54.063171 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:54.064561 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:54.065079 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:54.065178 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:54.066080 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:54.066087 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:54.066098 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:54.066603 (+ 505us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:54.066645 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:54.066662 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:54.066745 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:54.066946 (+ 201us) server_negotiation.cc:299] Negotiation successful
0925 09:27:54.067144 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":274,"threads_started":1}
I20260925 09:27:54.068701 20210 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:54.148658 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 20088
I20260925 09:27:54.165724 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 20219
I20260925 09:27:54.173359 20487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:54.166526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55158 (local address 127.10.114.66:35225)
0925 09:27:54.167083 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:54.167091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:54.167119 (+ 28us) server_negotiation.cc:407] Connection header received
0925 09:27:54.167179 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:54.167185 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:54.167407 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:54.167534 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:54.167984 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:54.169024 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:54.170246 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:54.170809 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:54.170898 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:54.171967 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:54.171974 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:54.171988 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:54.172548 (+ 560us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:54.172589 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:54.172607 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:54.172690 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:54.172881 (+ 191us) server_negotiation.cc:299] Negotiation successful
0925 09:27:54.173029 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":297,"threads_started":1}
I20260925 09:27:54.174525 20341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:54.252364 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 20219
I20260925 09:27:54.268436 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 20349
I20260925 09:27:54.276341 20489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:54.269456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43212 (local address 127.10.114.67:41945)
0925 09:27:54.270036 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:54.270042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:54.270072 (+ 30us) server_negotiation.cc:407] Connection header received
0925 09:27:54.270129 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:54.270135 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:54.270335 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:54.270470 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:54.270890 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:54.272027 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:54.273233 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:54.273855 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:54.273965 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:54.274959 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:54.274967 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:54.274980 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:54.275564 (+ 584us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:54.275617 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:54.275636 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:54.275725 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:54.275925 (+ 200us) server_negotiation.cc:299] Negotiation successful
0925 09:27:54.276043 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":315,"threads_started":1}
I20260925 09:27:54.277580 20471 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:54.357141 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 20349
I20260925 09:27:54.373245 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 19998
I20260925 09:27:54.374624 20059 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:54.480281 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 19998
2026-09-25T09:27:54Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2354 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-25T09:27:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:27:54Z Disabled control of system clock
I20260925 09:27:54.548943 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:32975
--webserver_interface=127.10.114.126
--webserver_port=0
--builtin_ntp_servers=127.10.114.84:45961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:32975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 09:27:54.864125 20495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:54.864400 20495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:54.864562 20495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:54.871434 20495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:27:54.871536 20495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:54.871660 20495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:54.871749 20495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:27:54.871852 20495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:27:54.881106 20495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:45961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:32975
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:32975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:54.882388 20495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:54.884392 20495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:54.896168 20500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:54.896201 20501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:54.896277 20503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:54.901134 20495 server_base.cc:1034] running on GCE node
I20260925 09:27:54.901849 20495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:54.903147 20495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:54.904412 20495 hybrid_clock.cc:648] HybridClock initialized: now 1790328474904360 us; error 55 us; skew 500 ppm
I20260925 09:27:54.904786 20495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:27:54.907106 20495 webserver.cc:460] Webserver started at http://127.10.114.126:44599/ using document root <none> and password file <none>
I20260925 09:27:54.907910 20495 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:54.908043 20495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:54.908375 20495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:54.911298 20495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "bf21b4a9441c40a3bc519706a0916f82"
format_stamp: "Formatted at 2026-09-25 09:27:54 on dist-test-slave-9pj8"
server_key: "1a2c02d2ab20a4add242dd7782b95e64"
server_key_iv: "e40ee42f2d032fc366907eda2c992975"
server_key_version: "encryptionkey@0"
I20260925 09:27:54.912192 20495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "bf21b4a9441c40a3bc519706a0916f82"
format_stamp: "Formatted at 2026-09-25 09:27:54 on dist-test-slave-9pj8"
server_key: "1a2c02d2ab20a4add242dd7782b95e64"
server_key_iv: "e40ee42f2d032fc366907eda2c992975"
server_key_version: "encryptionkey@0"
I20260925 09:27:54.917178 20495 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260925 09:27:54.920603 20509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:54.921804 20495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:27:54.922011 20495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "bf21b4a9441c40a3bc519706a0916f82"
format_stamp: "Formatted at 2026-09-25 09:27:54 on dist-test-slave-9pj8"
server_key: "1a2c02d2ab20a4add242dd7782b95e64"
server_key_iv: "e40ee42f2d032fc366907eda2c992975"
server_key_version: "encryptionkey@0"
I20260925 09:27:54.922233 20495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:54.928261 20495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:54.929023 20495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:54.929319 20495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:54.948467 20495 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:32975
I20260925 09:27:54.948518 20560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:32975 every 8 connection(s)
I20260925 09:27:54.949709 20495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:27:54.953253 20561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:54.955471 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 20495
I20260925 09:27:54.955689 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:27:54.956326 10697 external_mini_cluster.cc:1428] Setting key 300628f8810a8e87f868f75da893744e
I20260925 09:27:54.964867 20564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:54.958246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44624 (local address 127.10.114.126:32975)
0925 09:27:54.959266 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:54.959279 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:54.959322 (+ 43us) server_negotiation.cc:407] Connection header received
0925 09:27:54.961193 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:54.961230 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:54.961725 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:54.962110 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:54.963086 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:54.963110 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:54.963130 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:54.963145 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:54.963212 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:54.963231 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:54.963447 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:54.963563 (+ 116us) server_negotiation.cc:299] Negotiation successful
0925 09:27:54.963805 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":325,"threads_started":1}
I20260925 09:27:54.965377 20561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82: Bootstrap starting.
I20260925 09:27:54.968247 20561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:54.969141 20561 log.cc:824] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:54.971400 20561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82: No bootstrap required, opened a new log
I20260925 09:27:54.977880 20561 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf21b4a9441c40a3bc519706a0916f82" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 32975 } }
I20260925 09:27:54.978202 20561 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:54.978312 20561 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf21b4a9441c40a3bc519706a0916f82, State: Initialized, Role: FOLLOWER
I20260925 09:27:54.978796 20561 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf21b4a9441c40a3bc519706a0916f82" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 32975 } }
I20260925 09:27:54.979027 20561 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:27:54.979169 20561 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:27:54.979347 20561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:54.981338 20561 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf21b4a9441c40a3bc519706a0916f82" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 32975 } }
I20260925 09:27:54.981817 20561 leader_election.cc:302] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf21b4a9441c40a3bc519706a0916f82; no voters:
I20260925 09:27:54.982343 20561 leader_election.cc:288] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:27:54.982537 20566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:54.983189 20566 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82 [term 1 LEADER]: Becoming Leader. State: Replica: bf21b4a9441c40a3bc519706a0916f82, State: Running, Role: LEADER
I20260925 09:27:54.983652 20566 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf21b4a9441c40a3bc519706a0916f82" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 32975 } }
I20260925 09:27:54.984350 20561 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:27:54.986279 20568 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf21b4a9441c40a3bc519706a0916f82. Latest consensus state: current_term: 1 leader_uuid: "bf21b4a9441c40a3bc519706a0916f82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf21b4a9441c40a3bc519706a0916f82" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 32975 } } }
I20260925 09:27:54.986599 20568 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:54.986689 20567 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf21b4a9441c40a3bc519706a0916f82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf21b4a9441c40a3bc519706a0916f82" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 32975 } } }
I20260925 09:27:54.987044 20567 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:54.988471 20572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:27:54.992987 20572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:27:55.001731 20572 catalog_manager.cc:1347] Generated new cluster ID: 61224e75a5584f0aa82448b700bc4441
I20260925 09:27:55.001852 20572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:27:55.035023 20572 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:27:55.035472 20572 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:27:55.042301 20572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf21b4a9441c40a3bc519706a0916f82: Generated new TSK 0
I20260925 09:27:55.043238 20572 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:27:55.052268 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:32975
--builtin_ntp_servers=127.10.114.84:45961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 09:27:55.375353 20585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:55.375630 20585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:55.375720 20585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:55.382762 20585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:55.382855 20585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:55.382965 20585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:27:55.391539 20585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:45961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:32975
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:55.392789 20585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:55.394537 20585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:55.406744 20590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:55.406838 20591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:55.406805 20593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:55.407725 20585 server_base.cc:1034] running on GCE node
I20260925 09:27:55.408247 20585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:55.409061 20585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:55.410243 20585 hybrid_clock.cc:648] HybridClock initialized: now 1790328475410209 us; error 26 us; skew 500 ppm
I20260925 09:27:55.410516 20585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:27:55.412599 20585 webserver.cc:460] Webserver started at http://127.10.114.65:46717/ using document root <none> and password file <none>
I20260925 09:27:55.413192 20585 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:55.413262 20585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:55.413486 20585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:55.416004 20585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "052ff227999743d89ad3803620d027ad"
format_stamp: "Formatted at 2026-09-25 09:27:55 on dist-test-slave-9pj8"
server_key: "43a49b6e3df73621894f27598094bbcf"
server_key_iv: "5406ebb453aa4cfc52670227a77caf89"
server_key_version: "encryptionkey@0"
I20260925 09:27:55.416666 20585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "052ff227999743d89ad3803620d027ad"
format_stamp: "Formatted at 2026-09-25 09:27:55 on dist-test-slave-9pj8"
server_key: "43a49b6e3df73621894f27598094bbcf"
server_key_iv: "5406ebb453aa4cfc52670227a77caf89"
server_key_version: "encryptionkey@0"
I20260925 09:27:55.421142 20585 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:27:55.424151 20599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:55.425304 20585 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260925 09:27:55.425415 20585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "052ff227999743d89ad3803620d027ad"
format_stamp: "Formatted at 2026-09-25 09:27:55 on dist-test-slave-9pj8"
server_key: "43a49b6e3df73621894f27598094bbcf"
server_key_iv: "5406ebb453aa4cfc52670227a77caf89"
server_key_version: "encryptionkey@0"
I20260925 09:27:55.425637 20585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:55.431244 20585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:55.431903 20585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:55.432132 20585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:55.432931 20585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:55.434078 20585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:55.434151 20585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:55.434222 20585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:55.434269 20585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:55.465392 20585 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:45231
I20260925 09:27:55.465425 20711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:45231 every 8 connection(s)
I20260925 09:27:55.466572 20585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:27:55.471973 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 20585
I20260925 09:27:55.472182 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:27:55.472867 10697 external_mini_cluster.cc:1428] Setting key 698eb14417dd1c0ba3650d73aabe91e5
I20260925 09:27:55.474388 20714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:55.469151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:44409 (local address 127.10.114.126:32975)
0925 09:27:55.469599 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:55.469604 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:55.470622 (+ 1018us) server_negotiation.cc:407] Connection header received
0925 09:27:55.472260 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:55.472265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:55.472326 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:55.472445 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:55.473737 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:55.473742 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:55.473746 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:55.473762 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:55.473810 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:55.473831 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:55.473917 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:55.474137 (+ 220us) server_negotiation.cc:299] Negotiation successful
0925 09:27:55.474178 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":228,"threads_started":1}
I20260925 09:27:55.475373 20715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:55.469537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:32975 (local address 127.10.114.65:44409)
0925 09:27:55.470460 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:55.470500 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:55.471977 (+ 1477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:55.472632 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:55.472639 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:55.472720 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:55.473427 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:27:55.473430 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:55.473478 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:27:55.473587 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:55.473937 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:55.473948 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:55.473954 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 09:27:55.474215 (+ 261us) client_negotiation.cc:240] Negotiation successful
0925 09:27:55.474437 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":257,"threads_started":1}
I20260925 09:27:55.475733 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:32975
--builtin_ntp_servers=127.10.114.84:45961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260925 09:27:55.476797 20712 heartbeater.cc:347] Connected to a master server at 127.10.114.126:32975
I20260925 09:27:55.477085 20712 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:55.477571 20712 heartbeater.cc:511] Master 127.10.114.126:32975 requested a full tablet report, sending...
I20260925 09:27:55.479358 20526 ts_manager.cc:194] Registered new tserver with Master: 052ff227999743d89ad3803620d027ad (127.10.114.65:45231)
W20260925 09:27:55.785136 20716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:55.785434 20716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:55.785590 20716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:55.792275 20716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:55.792394 20716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:55.792560 20716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:27:55.801165 20716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:45961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:32975
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:55.802323 20716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:55.804047 20716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:55.816321 20721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:55.816437 20724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:55.816440 20722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:55.817625 20716 server_base.cc:1034] running on GCE node
I20260925 09:27:55.818176 20716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:55.819047 20716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:55.820317 20716 hybrid_clock.cc:648] HybridClock initialized: now 1790328475820274 us; error 58 us; skew 500 ppm
I20260925 09:27:55.820719 20716 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:27:55.823025 20716 webserver.cc:460] Webserver started at http://127.10.114.66:34719/ using document root <none> and password file <none>
I20260925 09:27:55.823705 20716 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:55.823884 20716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:55.824208 20716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:55.828806 20716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "8a667382544043e5bae0e3c5bf86287e"
format_stamp: "Formatted at 2026-09-25 09:27:55 on dist-test-slave-9pj8"
server_key: "e12076363ca9fa21e341d1927ab18bec"
server_key_iv: "3daf23692c20accfe077e37a97a90596"
server_key_version: "encryptionkey@0"
I20260925 09:27:55.829679 20716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "8a667382544043e5bae0e3c5bf86287e"
format_stamp: "Formatted at 2026-09-25 09:27:55 on dist-test-slave-9pj8"
server_key: "e12076363ca9fa21e341d1927ab18bec"
server_key_iv: "3daf23692c20accfe077e37a97a90596"
server_key_version: "encryptionkey@0"
I20260925 09:27:55.834515 20716 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 09:27:55.837953 20730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:55.839353 20716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260925 09:27:55.839509 20716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "8a667382544043e5bae0e3c5bf86287e"
format_stamp: "Formatted at 2026-09-25 09:27:55 on dist-test-slave-9pj8"
server_key: "e12076363ca9fa21e341d1927ab18bec"
server_key_iv: "3daf23692c20accfe077e37a97a90596"
server_key_version: "encryptionkey@0"
I20260925 09:27:55.839696 20716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:55.844810 20716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:55.845587 20716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:55.845836 20716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:55.846690 20716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:55.848081 20716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:55.848165 20716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:55.848304 20716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:55.848392 20716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:55.877447 20716 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:45443
I20260925 09:27:55.877497 20842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:45443 every 8 connection(s)
I20260925 09:27:55.878727 20716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:27:55.885730 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 20716
I20260925 09:27:55.885918 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:27:55.886425 10697 external_mini_cluster.cc:1428] Setting key cb0a5c1c1683d00bc96bfbb8509ba1c6
I20260925 09:27:55.886979 20714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:55.881168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:50675 (local address 127.10.114.126:32975)
0925 09:27:55.881402 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:55.881407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:55.882809 (+ 1402us) server_negotiation.cc:407] Connection header received
0925 09:27:55.884601 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:55.884605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:55.884654 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:55.884740 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:55.886165 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:55.886172 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:55.886177 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:55.886196 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:55.886235 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:55.886249 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:55.886370 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:55.886616 (+ 246us) server_negotiation.cc:299] Negotiation successful
0925 09:27:55.886678 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260925 09:27:55.888047 20845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:55.881721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:32975 (local address 127.10.114.66:50675)
0925 09:27:55.882657 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:55.882693 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:55.884362 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:55.884987 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:55.884995 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:55.885081 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:55.885797 (+ 716us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:27:55.885799 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:55.885874 (+ 75us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:27:55.885975 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:55.886373 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:55.886382 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:55.886389 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 09:27:55.886679 (+ 290us) client_negotiation.cc:240] Negotiation successful
0925 09:27:55.886906 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":275,"threads_started":1}
I20260925 09:27:55.889354 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:32975
--builtin_ntp_servers=127.10.114.84:45961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260925 09:27:55.889482 20843 heartbeater.cc:347] Connected to a master server at 127.10.114.126:32975
I20260925 09:27:55.889743 20843 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:55.890331 20843 heartbeater.cc:511] Master 127.10.114.126:32975 requested a full tablet report, sending...
I20260925 09:27:55.891695 20526 ts_manager.cc:194] Registered new tserver with Master: 8a667382544043e5bae0e3c5bf86287e (127.10.114.66:45443)
W20260925 09:27:56.211545 20846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:56.211898 20846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:56.212080 20846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:56.219264 20846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:56.219373 20846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:56.219552 20846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:27:56.228881 20846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:45961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:32975
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:56.230235 20846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:56.232102 20846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:56.245318 20852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:56.245246 20851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:56.245205 20854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:56.246089 20846 server_base.cc:1034] running on GCE node
I20260925 09:27:56.246656 20846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:56.247538 20846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:56.248792 20846 hybrid_clock.cc:648] HybridClock initialized: now 1790328476248714 us; error 58 us; skew 500 ppm
I20260925 09:27:56.249102 20846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:27:56.258924 20846 webserver.cc:460] Webserver started at http://127.10.114.67:36731/ using document root <none> and password file <none>
I20260925 09:27:56.259604 20846 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:56.259734 20846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:56.260084 20846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:56.262796 20846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "d882936c0956404ba992f9eb8e089056"
format_stamp: "Formatted at 2026-09-25 09:27:56 on dist-test-slave-9pj8"
server_key: "d92e34899ef06f9263ab5f42f9087bb4"
server_key_iv: "efa5ceb8d39603762c065338e18fb933"
server_key_version: "encryptionkey@0"
I20260925 09:27:56.263586 20846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "d882936c0956404ba992f9eb8e089056"
format_stamp: "Formatted at 2026-09-25 09:27:56 on dist-test-slave-9pj8"
server_key: "d92e34899ef06f9263ab5f42f9087bb4"
server_key_iv: "efa5ceb8d39603762c065338e18fb933"
server_key_version: "encryptionkey@0"
I20260925 09:27:56.268592 20846 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260925 09:27:56.271888 20860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:56.273016 20846 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 09:27:56.273169 20846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "d882936c0956404ba992f9eb8e089056"
format_stamp: "Formatted at 2026-09-25 09:27:56 on dist-test-slave-9pj8"
server_key: "d92e34899ef06f9263ab5f42f9087bb4"
server_key_iv: "efa5ceb8d39603762c065338e18fb933"
server_key_version: "encryptionkey@0"
I20260925 09:27:56.273356 20846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:56.278908 20846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:56.279700 20846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:56.279985 20846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:56.280794 20846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:56.281980 20846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:56.282101 20846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:56.282219 20846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:56.282322 20846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:56.313431 20846 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:45775
I20260925 09:27:56.313510 20972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:45775 every 8 connection(s)
I20260925 09:27:56.314642 20846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:27:56.320111 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 20846
I20260925 09:27:56.320298 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:27:56.320760 10697 external_mini_cluster.cc:1428] Setting key f3041ea3b4da45b849817568d322519e
I20260925 09:27:56.322469 20714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:56.317057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:52039 (local address 127.10.114.126:32975)
0925 09:27:56.317216 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:56.317220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:56.318509 (+ 1289us) server_negotiation.cc:407] Connection header received
0925 09:27:56.320246 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:56.320251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:56.320322 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:56.320461 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:56.321787 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:56.321799 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:56.321802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:56.321816 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:56.321843 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:56.321854 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:56.321978 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:56.322159 (+ 181us) server_negotiation.cc:299] Negotiation successful
0925 09:27:56.322230 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 09:27:56.323261 20975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:56.317539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:32975 (local address 127.10.114.67:52039)
0925 09:27:56.318367 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:56.318399 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:56.320005 (+ 1606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:56.320662 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:56.320669 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:56.320748 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:56.321496 (+ 748us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:27:56.321501 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:56.321549 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:27:56.321638 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:56.321953 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:56.321964 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:56.321972 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 09:27:56.322245 (+ 273us) client_negotiation.cc:240] Negotiation successful
0925 09:27:56.322454 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":237,"threads_started":1}
I20260925 09:27:56.324415 20973 heartbeater.cc:347] Connected to a master server at 127.10.114.126:32975
I20260925 09:27:56.324724 20973 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:56.325302 20973 heartbeater.cc:511] Master 127.10.114.126:32975 requested a full tablet report, sending...
I20260925 09:27:56.326671 20526 ts_manager.cc:194] Registered new tserver with Master: d882936c0956404ba992f9eb8e089056 (127.10.114.67:45775)
I20260925 09:27:56.335817 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 09:27:56.341374 20714 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:27:56.339910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44634 (local address 127.10.114.126:32975)
0925 09:27:56.340085 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:56.340092 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:56.340633 (+ 541us) server_negotiation.cc:407] Connection header received
0925 09:27:56.340767 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:56.340771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:56.340814 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:56.340881 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:56.341156 (+ 275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44634: BlockingRecv error: recv got EOF from 127.0.0.1:44634 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 09:27:56.342144 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 20585
I20260925 09:27:56.345604 20982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:56.343037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35078 (local address 127.10.114.65:45231)
0925 09:27:56.343571 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:56.343577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:56.343601 (+ 24us) server_negotiation.cc:407] Connection header received
0925 09:27:56.343662 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:56.343668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:56.343947 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:56.344075 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:56.344938 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:56.344945 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:56.344954 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:56.344967 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:56.345005 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:56.345031 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:56.345136 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:56.345254 (+ 118us) server_negotiation.cc:299] Negotiation successful
0925 09:27:56.345343 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":262,"threads_started":1}
I20260925 09:27:56.346776 20707 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:56.422701 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 20585
I20260925 09:27:56.438411 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 20716
I20260925 09:27:56.441819 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:56.439363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42936 (local address 127.10.114.66:45443)
0925 09:27:56.439901 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:56.439915 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:56.439940 (+ 25us) server_negotiation.cc:407] Connection header received
0925 09:27:56.439999 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:56.440005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:56.440208 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:56.440323 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:56.441155 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:56.441161 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:56.441178 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:56.441193 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:56.441230 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:56.441243 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:56.441334 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:56.441461 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 09:27:56.441575 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":245,"threads_started":1}
I20260925 09:27:56.442900 20838 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:56.521641 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 20716
I20260925 09:27:56.538425 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 20846
I20260925 09:27:56.542068 20986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:56.539420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45098 (local address 127.10.114.67:45775)
0925 09:27:56.540086 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:56.540093 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:56.540121 (+ 28us) server_negotiation.cc:407] Connection header received
0925 09:27:56.540185 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:56.540192 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:56.540411 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:56.540568 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:56.541383 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:56.541389 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:56.541399 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:27:56.541424 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:56.541459 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:27:56.541472 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:56.541589 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:56.541640 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 09:27:56.541746 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":392,"threads_started":1}
I20260925 09:27:56.543355 20968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:56.620004 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 20846
I20260925 09:27:56.635082 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 20495
I20260925 09:27:56.636453 20556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:27:56.731179 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 20495
2026-09-25T09:27:56Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2249 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:27:56 dist-test-slave-9pj8 krb5kdc[20991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:27:56 dist-test-slave-9pj8 krb5kdc[20991](info): set up 2 sockets
Sep 25 09:27:56 dist-test-slave-9pj8 krb5kdc[20991](info): commencing operation
krb5kdc: starting...
W20260925 09:27:57.784790 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.002s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:27:57 dist-test-slave-9pj8 krb5kdc[20991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:27:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:27:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:27:57.991221 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:35773
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:43313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:35773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:58.293329 21004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:58.293608 21004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:58.293773 21004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:58.300740 21004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:27:58.300828 21004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:58.300958 21004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:58.301056 21004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:27:58.301152 21004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:27:58.309921 21004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:43313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:35773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:35773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:58.311053 21004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:58.312256 21004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:58.323179 21009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:58.323134 21010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:58.323099 21012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:58.323825 21004 server_base.cc:1034] running on GCE node
I20260925 09:27:58.324602 21004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:58.325766 21004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:58.326987 21004 hybrid_clock.cc:648] HybridClock initialized: now 1790328478326953 us; error 43 us; skew 500 ppm
I20260925 09:27:58.327322 21004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:58 dist-test-slave-9pj8 krb5kdc[20991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328478, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:58.330345 21004 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:27:58.331741 21004 webserver.cc:460] Webserver started at http://127.10.114.126:39151/ using document root <none> and password file <none>
I20260925 09:27:58.332471 21004 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:58.332623 21004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:58.332897 21004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:58.335557 21004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "6475b02932c5465b8693fecc3b03b895"
format_stamp: "Formatted at 2026-09-25 09:27:58 on dist-test-slave-9pj8"
server_key: "069027f52c6364f5f112180e620f46fd"
server_key_iv: "4d1fab1cfeee5e047e4b9aac31525438"
server_key_version: "encryptionkey@0"
I20260925 09:27:58.336274 21004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "6475b02932c5465b8693fecc3b03b895"
format_stamp: "Formatted at 2026-09-25 09:27:58 on dist-test-slave-9pj8"
server_key: "069027f52c6364f5f112180e620f46fd"
server_key_iv: "4d1fab1cfeee5e047e4b9aac31525438"
server_key_version: "encryptionkey@0"
I20260925 09:27:58.340875 21004 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:27:58.344291 21019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:58.345561 21004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260925 09:27:58.345747 21004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "6475b02932c5465b8693fecc3b03b895"
format_stamp: "Formatted at 2026-09-25 09:27:58 on dist-test-slave-9pj8"
server_key: "069027f52c6364f5f112180e620f46fd"
server_key_iv: "4d1fab1cfeee5e047e4b9aac31525438"
server_key_version: "encryptionkey@0"
I20260925 09:27:58.345933 21004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:58.357168 21004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:58.361655 21004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:58.361912 21004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:58.378530 21004 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:35773
I20260925 09:27:58.378559 21070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:35773 every 8 connection(s)
I20260925 09:27:58.379861 21004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:27:58.383107 21071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:27:58.387876 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 21004
I20260925 09:27:58.388095 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:27:58.388696 10697 external_mini_cluster.cc:1428] Setting key 2cba0ddf06494edfdb38322448256cd7
I20260925 09:27:58.392722 21071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895: Bootstrap starting.
I20260925 09:27:58.396097 21071 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895: Neither blocks nor log segments found. Creating new log.
I20260925 09:27:58.397085 21071 log.cc:824] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895: Log is configured to *not* fsync() on all Append() calls
I20260925 09:27:58.399847 21071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895: No bootstrap required, opened a new log
Sep 25 09:27:58 dist-test-slave-9pj8 krb5kdc[20991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:27:58.407474 21071 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6475b02932c5465b8693fecc3b03b895" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 35773 } }
I20260925 09:27:58.407820 21071 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:27:58.407944 21071 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6475b02932c5465b8693fecc3b03b895, State: Initialized, Role: FOLLOWER
I20260925 09:27:58.408571 21071 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6475b02932c5465b8693fecc3b03b895" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 35773 } }
I20260925 09:27:58.408793 21071 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:27:58.408926 21071 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:27:58.409119 21071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:27:58.410076 21074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:58.390462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46332 (local address 127.10.114.126:35773)
0925 09:27:58.391208 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:58.391217 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:58.391253 (+ 36us) server_negotiation.cc:407] Connection header received
0925 09:27:58.392700 (+ 1447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:58.392722 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:58.393084 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:58.393362 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:58.395146 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:58.396520 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:58.397981 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:58.398639 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:58.398765 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:58.403102 (+ 4337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:58.403109 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:58.403124 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:58.403173 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:58.405968 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:58.406779 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:58.406795 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:58.406800 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:58.407009 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:58.407391 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:58.407397 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:58.407400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:58.408032 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:58.408325 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:58.408745 (+ 420us) server_negotiation.cc:299] Negotiation successful
0925 09:27:58.409141 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":350,"threads_started":1}
I20260925 09:27:58.411270 21071 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6475b02932c5465b8693fecc3b03b895" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 35773 } }
I20260925 09:27:58.411708 21071 leader_election.cc:302] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6475b02932c5465b8693fecc3b03b895; no voters:
I20260925 09:27:58.412381 21071 leader_election.cc:288] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:27:58.412506 21076 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:27:58.413173 21076 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895 [term 1 LEADER]: Becoming Leader. State: Replica: 6475b02932c5465b8693fecc3b03b895, State: Running, Role: LEADER
I20260925 09:27:58.413717 21076 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6475b02932c5465b8693fecc3b03b895" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 35773 } }
I20260925 09:27:58.414417 21071 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:27:58.416538 21078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6475b02932c5465b8693fecc3b03b895. Latest consensus state: current_term: 1 leader_uuid: "6475b02932c5465b8693fecc3b03b895" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6475b02932c5465b8693fecc3b03b895" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 35773 } } }
I20260925 09:27:58.416432 21077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6475b02932c5465b8693fecc3b03b895" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6475b02932c5465b8693fecc3b03b895" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 35773 } } }
I20260925 09:27:58.416887 21077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:58.419926 21078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895 [sys.catalog]: This master's current role is: LEADER
I20260925 09:27:58.426625 21083 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:27:58.431507 21083 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:27:58.439105 21083 catalog_manager.cc:1347] Generated new cluster ID: ad492da55db641ce939a1900ddc53206
I20260925 09:27:58.439227 21083 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:27:58.469782 21083 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:27:58.471483 21083 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:27:58.484782 21083 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895: Generated new TSK 0
I20260925 09:27:58.485673 21083 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
I20260925 09:27:58.539090 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:35773
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:43313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:58.844921 21097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:58.845158 21097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:58.845314 21097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:58.852190 21097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:58.852295 21097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:58.852471 21097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:27:58.861415 21097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:43313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:35773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:58.862545 21097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:58.863746 21097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:58.876298 21102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:58.876322 21103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:58.876780 21105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:58.877089 21097 server_base.cc:1034] running on GCE node
I20260925 09:27:58.877719 21097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:58.878571 21097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:58.879843 21097 hybrid_clock.cc:648] HybridClock initialized: now 1790328478879759 us; error 51 us; skew 500 ppm
I20260925 09:27:58.880162 21097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:58 dist-test-slave-9pj8 krb5kdc[20991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328478, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:58.883116 21097 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:27:58.884547 21097 webserver.cc:460] Webserver started at http://127.10.114.65:37781/ using document root <none> and password file <none>
I20260925 09:27:58.885222 21097 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:58.885363 21097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:58.885669 21097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:58.888494 21097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "ef1ea2b64acd41999135a613387925c9"
format_stamp: "Formatted at 2026-09-25 09:27:58 on dist-test-slave-9pj8"
server_key: "7d3a0ce0b9dfc5141b0cb01781ccdf40"
server_key_iv: "7d30ac9d1a27a95f5aedfbd17883f7ca"
server_key_version: "encryptionkey@0"
I20260925 09:27:58.889222 21097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "ef1ea2b64acd41999135a613387925c9"
format_stamp: "Formatted at 2026-09-25 09:27:58 on dist-test-slave-9pj8"
server_key: "7d3a0ce0b9dfc5141b0cb01781ccdf40"
server_key_iv: "7d30ac9d1a27a95f5aedfbd17883f7ca"
server_key_version: "encryptionkey@0"
I20260925 09:27:58.894001 21097 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:27:58.897272 21112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:58.898639 21097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 09:27:58.898801 21097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "ef1ea2b64acd41999135a613387925c9"
format_stamp: "Formatted at 2026-09-25 09:27:58 on dist-test-slave-9pj8"
server_key: "7d3a0ce0b9dfc5141b0cb01781ccdf40"
server_key_iv: "7d30ac9d1a27a95f5aedfbd17883f7ca"
server_key_version: "encryptionkey@0"
I20260925 09:27:58.899039 21097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:58.922016 21097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:58.926457 21097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:58.926795 21097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:58.927815 21097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:58.929141 21097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:58.929221 21097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:58.929405 21097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:58.929474 21097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:58.962340 21097 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:44751
I20260925 09:27:58.962379 21224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:44751 every 8 connection(s)
I20260925 09:27:58.963550 21097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:27:58.969115 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 21097
I20260925 09:27:58.969359 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:27:58.969923 10697 external_mini_cluster.cc:1428] Setting key 571026ca93f5ef3e31269a3dabe6f56a
Sep 25 09:27:58 dist-test-slave-9pj8 krb5kdc[20991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328478, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:27:58.983530 21227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:58.965948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:58973 (local address 127.10.114.126:35773)
0925 09:27:58.966375 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:58.966380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:58.967459 (+ 1079us) server_negotiation.cc:407] Connection header received
0925 09:27:58.969534 (+ 2075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:58.969543 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:58.969641 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:58.969767 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:58.971389 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:58.972312 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:58.973877 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:58.974304 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:58.974471 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:58.978000 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:58.978005 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:58.978008 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:58.978040 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:58.980681 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:58.981374 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:58.981381 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:58.981384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:58.981506 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:58.981925 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:58.981930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:58.981933 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:58.982192 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:58.982340 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:58.982958 (+ 618us) server_negotiation.cc:299] Negotiation successful
0925 09:27:58.983153 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":214,"threads_started":1}
I20260925 09:27:58.984530 21228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:58.966538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:35773 (local address 127.10.114.65:58973)
0925 09:27:58.967301 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:58.967345 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:58.969167 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:58.969999 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:58.970011 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:58.970629 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:58.971225 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:58.971242 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:58.972476 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:58.972481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:58.973722 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:58.973735 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:58.974563 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:58.974573 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:58.974835 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:58.975575 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:58.975599 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:58.977776 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:58.980814 (+ 3038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:58.980822 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:58.980837 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:58.981257 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:58.981629 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:58.981634 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:58.981638 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:58.981808 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:58.982315 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:58.982323 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:58.982634 (+ 311us) client_negotiation.cc:769] Sending connection context
0925 09:27:58.982891 (+ 257us) client_negotiation.cc:240] Negotiation successful
0925 09:27:58.983364 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":311,"threads_started":1}
I20260925 09:27:58.985977 21225 heartbeater.cc:347] Connected to a master server at 127.10.114.126:35773
I20260925 09:27:58.986279 21225 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:58.987134 21225 heartbeater.cc:511] Master 127.10.114.126:35773 requested a full tablet report, sending...
I20260925 09:27:58.989612 21036 ts_manager.cc:194] Registered new tserver with Master: ef1ea2b64acd41999135a613387925c9 (127.10.114.65:44751)
WARNING: no policy specified for kudu/127.10.114.66@KRBTEST.COM; defaulting to no policy
I20260925 09:27:58.991668 21036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:58973
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
I20260925 09:27:59.019866 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:35773
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:43313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:59.330420 21231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:59.330705 21231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:59.330875 21231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:59.337926 21231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:59.338066 21231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:59.338255 21231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:27:59.347198 21231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:43313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:35773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:59.348433 21231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:59.349769 21231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:59.362402 21239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:59.362532 21236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:59.362556 21237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:59.363430 21231 server_base.cc:1034] running on GCE node
I20260925 09:27:59.364107 21231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:59.364980 21231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:59.366217 21231 hybrid_clock.cc:648] HybridClock initialized: now 1790328479366145 us; error 60 us; skew 500 ppm
I20260925 09:27:59.366528 21231 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:59 dist-test-slave-9pj8 krb5kdc[20991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328479, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:59.369691 21231 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:27:59.371073 21231 webserver.cc:460] Webserver started at http://127.10.114.66:46179/ using document root <none> and password file <none>
I20260925 09:27:59.371706 21231 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:59.371871 21231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:59.372171 21231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:59.374738 21231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "4c88127f42e748a3b0e70fc2f1d19318"
format_stamp: "Formatted at 2026-09-25 09:27:59 on dist-test-slave-9pj8"
server_key: "f205caecf6b211a67d74a1dd3667c986"
server_key_iv: "77e6b69f9dd7bc8b7404c082134a535d"
server_key_version: "encryptionkey@0"
I20260925 09:27:59.375463 21231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "4c88127f42e748a3b0e70fc2f1d19318"
format_stamp: "Formatted at 2026-09-25 09:27:59 on dist-test-slave-9pj8"
server_key: "f205caecf6b211a67d74a1dd3667c986"
server_key_iv: "77e6b69f9dd7bc8b7404c082134a535d"
server_key_version: "encryptionkey@0"
I20260925 09:27:59.380430 21231 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 09:27:59.383603 21246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:59.384835 21231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:27:59.384999 21231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "4c88127f42e748a3b0e70fc2f1d19318"
format_stamp: "Formatted at 2026-09-25 09:27:59 on dist-test-slave-9pj8"
server_key: "f205caecf6b211a67d74a1dd3667c986"
server_key_iv: "77e6b69f9dd7bc8b7404c082134a535d"
server_key_version: "encryptionkey@0"
I20260925 09:27:59.385172 21231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:59.413844 21231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:59.417599 21231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:59.417876 21231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:59.418751 21231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:59.420094 21231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:59.420192 21231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:59.420346 21231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:59.420423 21231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:59.452965 21231 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:41945
I20260925 09:27:59.453044 21358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:41945 every 8 connection(s)
I20260925 09:27:59.454160 21231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:27:59.459600 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 21231
I20260925 09:27:59.459839 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:27:59.460422 10697 external_mini_cluster.cc:1428] Setting key d82fe0c6dc983b8c575e8bf71c4de3ac
Sep 25 09:27:59 dist-test-slave-9pj8 krb5kdc[20991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328479, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:27:59.474795 21227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:59.456753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:54477 (local address 127.10.114.126:35773)
0925 09:27:59.456931 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:59.456935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:59.457999 (+ 1064us) server_negotiation.cc:407] Connection header received
0925 09:27:59.460196 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:59.460201 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:59.460267 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:59.460344 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:59.462185 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:59.462909 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:59.464413 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:59.464774 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:59.464895 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:59.468549 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:59.468554 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:59.468557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:59.468585 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:59.471943 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:59.472574 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:59.472581 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:59.472587 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:59.472730 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:59.473123 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:59.473130 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:59.473153 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:59.473489 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:59.473636 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:59.474188 (+ 552us) server_negotiation.cc:299] Negotiation successful
0925 09:27:59.474413 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:27:59.475332 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:59.457174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:35773 (local address 127.10.114.66:54477)
0925 09:27:59.457849 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:59.457887 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:59.459893 (+ 2006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:59.460646 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:59.460657 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:59.461257 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:59.462007 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:59.462033 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:59.463060 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:59.463072 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:59.464276 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:59.464288 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:59.464967 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:59.464997 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:59.465216 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:59.465815 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:59.465833 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:59.468352 (+ 2519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:59.472074 (+ 3722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:59.472080 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:59.472090 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:59.472443 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:59.472832 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:59.472835 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:59.472837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:59.473007 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:59.473592 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:59.473597 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:59.473852 (+ 255us) client_negotiation.cc:769] Sending connection context
0925 09:27:59.474046 (+ 194us) client_negotiation.cc:240] Negotiation successful
0925 09:27:59.474363 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":276,"threads_started":1}
I20260925 09:27:59.476728 21359 heartbeater.cc:347] Connected to a master server at 127.10.114.126:35773
I20260925 09:27:59.477012 21359 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:59.477631 21359 heartbeater.cc:511] Master 127.10.114.126:35773 requested a full tablet report, sending...
I20260925 09:27:59.478981 21036 ts_manager.cc:194] Registered new tserver with Master: 4c88127f42e748a3b0e70fc2f1d19318 (127.10.114.66:41945)
I20260925 09:27:59.480201 21036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:54477
WARNING: no policy specified for kudu/127.10.114.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
I20260925 09:27:59.508893 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:35773
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:43313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:27:59.827558 21364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:27:59.827901 21364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:27:59.828070 21364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:27:59.834945 21364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:27:59.835071 21364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:27:59.835300 21364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:27:59.844599 21364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:43313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:35773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:27:59.845762 21364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:27:59.847000 21364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:27:59.859998 21369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:59.860070 21370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:27:59.860112 21372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:27:59.860929 21364 server_base.cc:1034] running on GCE node
I20260925 09:27:59.861414 21364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:27:59.862178 21364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:27:59.863373 21364 hybrid_clock.cc:648] HybridClock initialized: now 1790328479863329 us; error 41 us; skew 500 ppm
I20260925 09:27:59.863703 21364 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:27:59 dist-test-slave-9pj8 krb5kdc[20991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328479, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:27:59.866770 21364 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:27:59.868111 21364 webserver.cc:460] Webserver started at http://127.10.114.67:40759/ using document root <none> and password file <none>
I20260925 09:27:59.868746 21364 fs_manager.cc:359] Metadata directory not provided
I20260925 09:27:59.868883 21364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:27:59.869160 21364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:27:59.871922 21364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "a590ad2ffd9b4920973fd61dab247917"
format_stamp: "Formatted at 2026-09-25 09:27:59 on dist-test-slave-9pj8"
server_key: "a11a66b64d925f6a7678ca1c6da5ec79"
server_key_iv: "7fd08d6fc71a9184102852dddd4d82cd"
server_key_version: "encryptionkey@0"
I20260925 09:27:59.872637 21364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "a590ad2ffd9b4920973fd61dab247917"
format_stamp: "Formatted at 2026-09-25 09:27:59 on dist-test-slave-9pj8"
server_key: "a11a66b64d925f6a7678ca1c6da5ec79"
server_key_iv: "7fd08d6fc71a9184102852dddd4d82cd"
server_key_version: "encryptionkey@0"
I20260925 09:27:59.877461 21364 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:27:59.880834 21379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:59.882100 21364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:27:59.882220 21364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "a590ad2ffd9b4920973fd61dab247917"
format_stamp: "Formatted at 2026-09-25 09:27:59 on dist-test-slave-9pj8"
server_key: "a11a66b64d925f6a7678ca1c6da5ec79"
server_key_iv: "7fd08d6fc71a9184102852dddd4d82cd"
server_key_version: "encryptionkey@0"
I20260925 09:27:59.882375 21364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:27:59.894104 21364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:27:59.898053 21364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:27:59.898299 21364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:27:59.899108 21364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:27:59.900328 21364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:27:59.900398 21364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:59.900475 21364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:27:59.900518 21364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:27:59.934082 21364 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:38735
I20260925 09:27:59.934119 21491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:38735 every 8 connection(s)
I20260925 09:27:59.935518 21364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:27:59.937340 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 21364
I20260925 09:27:59.937539 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:27:59.937987 10697 external_mini_cluster.cc:1428] Setting key 8b304c9c67b875405c52e036478fc653
Sep 25 09:27:59 dist-test-slave-9pj8 krb5kdc[20991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328479, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:27:59.955456 21227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:59.938078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:43111 (local address 127.10.114.126:35773)
0925 09:27:59.938296 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:59.938302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:59.939689 (+ 1387us) server_negotiation.cc:407] Connection header received
0925 09:27:59.941785 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:59.941789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:59.941848 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:59.941924 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:59.943509 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:59.944285 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:59.945722 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:59.946090 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:59.946196 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:59.949676 (+ 3480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:59.949680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:59.949684 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:59.949714 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:59.952672 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:59.953435 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:59.953439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:59.953442 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:59.953583 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:59.954000 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:59.954015 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:59.954017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:59.954274 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:59.954380 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:59.955073 (+ 693us) server_negotiation.cc:299] Negotiation successful
0925 09:27:59.955232 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 09:27:59.956637 21494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:59.938679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:35773 (local address 127.10.114.67:43111)
0925 09:27:59.939517 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0925 09:27:59.939566 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 09:27:59.941563 (+ 1997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:27:59.942211 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:27:59.942232 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:27:59.942751 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:27:59.943356 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:59.943375 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:59.944421 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:59.944425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:59.945608 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:27:59.945620 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:27:59.946300 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:27:59.946304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:27:59.946480 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:59.947153 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:27:59.947177 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:27:59.949460 (+ 2283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:27:59.952822 (+ 3362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:59.952829 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:59.952842 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:59.953301 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:59.953683 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:27:59.953686 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:27:59.953688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:27:59.953896 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:27:59.954382 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:27:59.954389 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:27:59.954747 (+ 358us) client_negotiation.cc:769] Sending connection context
0925 09:27:59.954987 (+ 240us) client_negotiation.cc:240] Negotiation successful
0925 09:27:59.955362 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":299,"threads_started":1}
I20260925 09:27:59.958811 21492 heartbeater.cc:347] Connected to a master server at 127.10.114.126:35773
I20260925 09:27:59.959169 21492 heartbeater.cc:464] Registering TS with master...
I20260925 09:27:59.960063 21492 heartbeater.cc:511] Master 127.10.114.126:35773 requested a full tablet report, sending...
I20260925 09:27:59.961509 21036 ts_manager.cc:194] Registered new tserver with Master: a590ad2ffd9b4920973fd61dab247917 (127.10.114.67:38735)
I20260925 09:27:59.962393 21036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:43111
I20260925 09:27:59.965816 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:27:59.981489 21227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:27:59.970091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46344 (local address 127.10.114.126:35773)
0925 09:27:59.970252 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 09:27:59.970257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:27:59.970707 (+ 450us) server_negotiation.cc:407] Connection header received
0925 09:27:59.970871 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:27:59.970874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:27:59.970933 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:27:59.971025 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:27:59.972357 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:59.973039 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:59.974161 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:27:59.974526 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:27:59.974616 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:27:59.976718 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:27:59.976723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:27:59.976726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:27:59.976759 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:27:59.979245 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:59.979957 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:59.979961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:59.979963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:59.980101 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:27:59.980418 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:27:59.980421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:27:59.980423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:27:59.980706 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:27:59.980867 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0925 09:27:59.981133 (+ 266us) server_negotiation.cc:299] Negotiation successful
0925 09:27:59.981284 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 09:27:59.985528 21036 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:27:59.992429 21036 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:27:59.994692 21225 heartbeater.cc:503] Master 127.10.114.126:35773 was elected leader, sending a full tablet report...
I20260925 09:28:00.015753 21506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.005112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52062 (local address 127.10.114.65:44751)
0925 09:28:00.005814 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:00.005823 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:00.006764 (+ 941us) server_negotiation.cc:407] Connection header received
0925 09:28:00.007170 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:00.007184 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:00.007470 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:00.007673 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:28:00.008708 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.010314 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.012888 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.014439 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.014576 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:00.014800 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:00.015227 (+ 427us) server_negotiation.cc:299] Negotiation successful
0925 09:28:00.015394 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":410,"threads_started":1}
I20260925 09:28:00.015874 21507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.004371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:44751 (local address 127.0.0.1:52062)
0925 09:28:00.006596 (+ 2225us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:00.006641 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 09:28:00.006859 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:28:00.007987 (+ 1128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:28:00.007994 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:28:00.008036 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:28:00.008452 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:00.008470 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.010474 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.010479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:00.012734 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:00.012745 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.014652 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.014659 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:00.014886 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:00.014950 (+ 64us) client_negotiation.cc:769] Sending connection context
0925 09:28:00.015149 (+ 199us) client_negotiation.cc:240] Negotiation successful
0925 09:28:00.015356 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2028,"mutex_wait_us":1,"thread_start_us":2897,"threads_started":1}
I20260925 09:28:00.016842 21504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.004419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:41945 (local address 127.0.0.1:53470)
0925 09:28:00.007588 (+ 3169us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:00.007607 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:28:00.007723 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:28:00.008463 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:28:00.008469 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:28:00.008495 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:28:00.008969 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:00.008983 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.010999 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.011009 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:00.013366 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:00.013382 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.016060 (+ 2678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.016068 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:00.016231 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:00.016265 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 09:28:00.016399 (+ 134us) client_negotiation.cc:240] Negotiation successful
0925 09:28:00.016523 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3019,"thread_start_us":1870,"threads_started":1}
I20260925 09:28:00.017027 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.004813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53470 (local address 127.10.114.66:41945)
0925 09:28:00.005407 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:00.005419 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:00.007887 (+ 2468us) server_negotiation.cc:407] Connection header received
0925 09:28:00.008023 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:00.008033 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:00.008258 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:00.008463 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:28:00.009193 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.010807 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.013547 (+ 2740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.014701 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.014883 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:00.015105 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:00.016503 (+ 1398us) server_negotiation.cc:299] Negotiation successful
0925 09:28:00.016688 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":277,"threads_started":1}
I20260925 09:28:00.018018 21508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.007666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:38735 (local address 127.0.0.1:48212)
0925 09:28:00.009675 (+ 2009us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:00.009693 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:28:00.009817 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:28:00.010364 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:28:00.010369 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:28:00.010394 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:28:00.010940 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:00.010976 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.013073 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.013078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:00.015131 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:00.015146 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.017299 (+ 2153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.017305 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:00.017472 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:00.017523 (+ 51us) client_negotiation.cc:769] Sending connection context
0925 09:28:00.017611 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 09:28:00.017736 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1898,"thread_start_us":1732,"threads_started":1}
I20260925 09:28:00.018996 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.008256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48212 (local address 127.10.114.67:38735)
0925 09:28:00.009224 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:00.009234 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:00.009804 (+ 570us) server_negotiation.cc:407] Connection header received
0925 09:28:00.009970 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:00.009977 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:00.010171 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:00.010360 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:28:00.011221 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.012919 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.015304 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.016774 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.016920 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:00.017169 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:00.018530 (+ 1361us) server_negotiation.cc:299] Negotiation successful
0925 09:28:00.018685 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":779,"thread_start_us":589,"threads_started":1}
I20260925 09:28:00.020591 21294 tablet_service.cc:1467] Processing CreateTablet for tablet 995b5361e422434491159ba09293a901 (DEFAULT_TABLE table=test-table [id=b2b00df78f32471fa841572165eb1636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:28:00.020579 21160 tablet_service.cc:1467] Processing CreateTablet for tablet 995b5361e422434491159ba09293a901 (DEFAULT_TABLE table=test-table [id=b2b00df78f32471fa841572165eb1636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:28:00.021929 21294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 995b5361e422434491159ba09293a901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:28:00.022012 21160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 995b5361e422434491159ba09293a901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:28:00.022338 21427 tablet_service.cc:1467] Processing CreateTablet for tablet 995b5361e422434491159ba09293a901 (DEFAULT_TABLE table=test-table [id=b2b00df78f32471fa841572165eb1636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:28:00.023674 21427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 995b5361e422434491159ba09293a901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:28:00.034716 21510 tablet_bootstrap.cc:492] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318: Bootstrap starting.
I20260925 09:28:00.035991 21511 tablet_bootstrap.cc:492] T 995b5361e422434491159ba09293a901 P ef1ea2b64acd41999135a613387925c9: Bootstrap starting.
I20260925 09:28:00.038658 21510 tablet_bootstrap.cc:654] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318: Neither blocks nor log segments found. Creating new log.
I20260925 09:28:00.039525 21512 tablet_bootstrap.cc:492] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917: Bootstrap starting.
I20260925 09:28:00.039620 21511 tablet_bootstrap.cc:654] T 995b5361e422434491159ba09293a901 P ef1ea2b64acd41999135a613387925c9: Neither blocks nor log segments found. Creating new log.
I20260925 09:28:00.040061 21510 log.cc:824] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318: Log is configured to *not* fsync() on all Append() calls
I20260925 09:28:00.041085 21511 log.cc:824] T 995b5361e422434491159ba09293a901 P ef1ea2b64acd41999135a613387925c9: Log is configured to *not* fsync() on all Append() calls
I20260925 09:28:00.042975 21512 tablet_bootstrap.cc:654] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917: Neither blocks nor log segments found. Creating new log.
I20260925 09:28:00.043581 21510 tablet_bootstrap.cc:492] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318: No bootstrap required, opened a new log
I20260925 09:28:00.043898 21510 ts_tablet_manager.cc:1397] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.007s
I20260925 09:28:00.043960 21511 tablet_bootstrap.cc:492] T 995b5361e422434491159ba09293a901 P ef1ea2b64acd41999135a613387925c9: No bootstrap required, opened a new log
I20260925 09:28:00.044132 21512 log.cc:824] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917: Log is configured to *not* fsync() on all Append() calls
I20260925 09:28:00.044246 21511 ts_tablet_manager.cc:1397] T 995b5361e422434491159ba09293a901 P ef1ea2b64acd41999135a613387925c9: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260925 09:28:00.047006 21512 tablet_bootstrap.cc:492] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917: No bootstrap required, opened a new log
I20260925 09:28:00.047228 21512 ts_tablet_manager.cc:1397] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260925 09:28:00.051738 21510 raft_consensus.cc:348] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c88127f42e748a3b0e70fc2f1d19318" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41945 } } peers { permanent_uuid: "ef1ea2b64acd41999135a613387925c9" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44751 } } peers { permanent_uuid: "a590ad2ffd9b4920973fd61dab247917" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38735 } }
I20260925 09:28:00.052206 21510 raft_consensus.cc:374] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:28:00.052335 21510 raft_consensus.cc:729] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c88127f42e748a3b0e70fc2f1d19318, State: Initialized, Role: FOLLOWER
I20260925 09:28:00.053097 21510 consensus_queue.cc:260] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c88127f42e748a3b0e70fc2f1d19318" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41945 } } peers { permanent_uuid: "ef1ea2b64acd41999135a613387925c9" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44751 } } peers { permanent_uuid: "a590ad2ffd9b4920973fd61dab247917" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38735 } }
I20260925 09:28:00.054346 21511 raft_consensus.cc:348] T 995b5361e422434491159ba09293a901 P ef1ea2b64acd41999135a613387925c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c88127f42e748a3b0e70fc2f1d19318" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41945 } } peers { permanent_uuid: "ef1ea2b64acd41999135a613387925c9" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44751 } } peers { permanent_uuid: "a590ad2ffd9b4920973fd61dab247917" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38735 } }
I20260925 09:28:00.054718 21359 heartbeater.cc:503] Master 127.10.114.126:35773 was elected leader, sending a full tablet report...
I20260925 09:28:00.054778 21511 raft_consensus.cc:374] T 995b5361e422434491159ba09293a901 P ef1ea2b64acd41999135a613387925c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:28:00.054867 21511 raft_consensus.cc:729] T 995b5361e422434491159ba09293a901 P ef1ea2b64acd41999135a613387925c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef1ea2b64acd41999135a613387925c9, State: Initialized, Role: FOLLOWER
I20260925 09:28:00.055402 21510 ts_tablet_manager.cc:1428] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318: Time spent starting tablet: real 0.011s user 0.005s sys 0.005s
I20260925 09:28:00.055506 21511 consensus_queue.cc:260] T 995b5361e422434491159ba09293a901 P ef1ea2b64acd41999135a613387925c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c88127f42e748a3b0e70fc2f1d19318" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41945 } } peers { permanent_uuid: "ef1ea2b64acd41999135a613387925c9" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44751 } } peers { permanent_uuid: "a590ad2ffd9b4920973fd61dab247917" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38735 } }
I20260925 09:28:00.059298 21512 raft_consensus.cc:348] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c88127f42e748a3b0e70fc2f1d19318" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41945 } } peers { permanent_uuid: "ef1ea2b64acd41999135a613387925c9" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44751 } } peers { permanent_uuid: "a590ad2ffd9b4920973fd61dab247917" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38735 } }
I20260925 09:28:00.059749 21512 raft_consensus.cc:374] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:28:00.059918 21512 raft_consensus.cc:729] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a590ad2ffd9b4920973fd61dab247917, State: Initialized, Role: FOLLOWER
I20260925 09:28:00.060371 21511 ts_tablet_manager.cc:1428] T 995b5361e422434491159ba09293a901 P ef1ea2b64acd41999135a613387925c9: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260925 09:28:00.060695 21512 consensus_queue.cc:260] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c88127f42e748a3b0e70fc2f1d19318" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41945 } } peers { permanent_uuid: "ef1ea2b64acd41999135a613387925c9" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44751 } } peers { permanent_uuid: "a590ad2ffd9b4920973fd61dab247917" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38735 } }
I20260925 09:28:00.061813 21492 heartbeater.cc:503] Master 127.10.114.126:35773 was elected leader, sending a full tablet report...
I20260925 09:28:00.062426 21512 ts_tablet_manager.cc:1428] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917: Time spent starting tablet: real 0.015s user 0.008s sys 0.007s
I20260925 09:28:00.132588 21516 raft_consensus.cc:482] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:28:00.132843 21516 raft_consensus.cc:504] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c88127f42e748a3b0e70fc2f1d19318" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41945 } } peers { permanent_uuid: "ef1ea2b64acd41999135a613387925c9" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44751 } } peers { permanent_uuid: "a590ad2ffd9b4920973fd61dab247917" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38735 } }
I20260925 09:28:00.134510 21516 leader_election.cc:288] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef1ea2b64acd41999135a613387925c9 (127.10.114.65:44751), a590ad2ffd9b4920973fd61dab247917 (127.10.114.67:38735)
I20260925 09:28:00.141453 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.136033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:46725 (local address 127.10.114.67:38735)
0925 09:28:00.136271 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:00.136277 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:00.136372 (+ 95us) server_negotiation.cc:407] Connection header received
0925 09:28:00.136511 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:00.136516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:00.136604 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:00.136711 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:28:00.137260 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.137970 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.139685 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.140494 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.140671 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:00.140752 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:00.141050 (+ 298us) server_negotiation.cc:299] Negotiation successful
0925 09:28:00.141187 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260925 09:28:00.141472 21506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.134865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:49473 (local address 127.10.114.65:44751)
0925 09:28:00.135073 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:00.135077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:00.135725 (+ 648us) server_negotiation.cc:407] Connection header received
0925 09:28:00.136091 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:00.136097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:00.136183 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:00.136247 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:28:00.137066 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.137816 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.139691 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.140573 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.140729 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:00.140799 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:00.141091 (+ 292us) server_negotiation.cc:299] Negotiation successful
0925 09:28:00.141224 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260925 09:28:00.141610 21519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.134871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:44751 (local address 127.10.114.66:49473)
0925 09:28:00.135634 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:00.135653 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:28:00.135875 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:28:00.136482 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:28:00.136487 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:28:00.136514 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:28:00.136916 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:00.136926 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.137942 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.137947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:00.139540 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:00.139552 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.140795 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.140807 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:00.140923 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:00.140943 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 09:28:00.141055 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 09:28:00.141188 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":39,"thread_start_us":395,"threads_started":1}
I20260925 09:28:00.141896 21520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.135675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:38735 (local address 127.10.114.66:46725)
0925 09:28:00.136276 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:00.136290 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 09:28:00.136396 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:28:00.136728 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:28:00.136730 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:28:00.136746 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:28:00.137113 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:00.137127 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.138100 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.138104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:00.139537 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:00.139548 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.140708 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.140715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:00.140891 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:00.140924 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 09:28:00.140994 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 09:28:00.141147 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":28,"thread_start_us":301,"threads_started":1}
I20260925 09:28:00.142562 21180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "995b5361e422434491159ba09293a901" candidate_uuid: "4c88127f42e748a3b0e70fc2f1d19318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef1ea2b64acd41999135a613387925c9" is_pre_election: true
I20260925 09:28:00.142740 21447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "995b5361e422434491159ba09293a901" candidate_uuid: "4c88127f42e748a3b0e70fc2f1d19318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a590ad2ffd9b4920973fd61dab247917" is_pre_election: true
I20260925 09:28:00.143082 21180 raft_consensus.cc:2454] T 995b5361e422434491159ba09293a901 P ef1ea2b64acd41999135a613387925c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c88127f42e748a3b0e70fc2f1d19318 in term 0.
I20260925 09:28:00.143250 21447 raft_consensus.cc:2454] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c88127f42e748a3b0e70fc2f1d19318 in term 0.
I20260925 09:28:00.143713 21250 leader_election.cc:302] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c88127f42e748a3b0e70fc2f1d19318, ef1ea2b64acd41999135a613387925c9; no voters:
I20260925 09:28:00.144095 21516 raft_consensus.cc:2781] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:28:00.144239 21516 raft_consensus.cc:482] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:28:00.144361 21516 raft_consensus.cc:3037] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:28:00.146497 21516 raft_consensus.cc:504] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c88127f42e748a3b0e70fc2f1d19318" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41945 } } peers { permanent_uuid: "ef1ea2b64acd41999135a613387925c9" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44751 } } peers { permanent_uuid: "a590ad2ffd9b4920973fd61dab247917" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38735 } }
I20260925 09:28:00.147006 21516 leader_election.cc:288] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [CANDIDATE]: Term 1 election: Requested vote from peers ef1ea2b64acd41999135a613387925c9 (127.10.114.65:44751), a590ad2ffd9b4920973fd61dab247917 (127.10.114.67:38735)
I20260925 09:28:00.147410 21180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "995b5361e422434491159ba09293a901" candidate_uuid: "4c88127f42e748a3b0e70fc2f1d19318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef1ea2b64acd41999135a613387925c9"
I20260925 09:28:00.147500 21447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "995b5361e422434491159ba09293a901" candidate_uuid: "4c88127f42e748a3b0e70fc2f1d19318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a590ad2ffd9b4920973fd61dab247917"
I20260925 09:28:00.147608 21180 raft_consensus.cc:3037] T 995b5361e422434491159ba09293a901 P ef1ea2b64acd41999135a613387925c9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:28:00.147696 21447 raft_consensus.cc:3037] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:28:00.149819 21180 raft_consensus.cc:2454] T 995b5361e422434491159ba09293a901 P ef1ea2b64acd41999135a613387925c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c88127f42e748a3b0e70fc2f1d19318 in term 1.
I20260925 09:28:00.149928 21447 raft_consensus.cc:2454] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c88127f42e748a3b0e70fc2f1d19318 in term 1.
I20260925 09:28:00.150223 21250 leader_election.cc:302] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c88127f42e748a3b0e70fc2f1d19318, ef1ea2b64acd41999135a613387925c9; no voters:
I20260925 09:28:00.150471 21516 raft_consensus.cc:2781] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:28:00.150880 21516 raft_consensus.cc:686] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [term 1 LEADER]: Becoming Leader. State: Replica: 4c88127f42e748a3b0e70fc2f1d19318, State: Running, Role: LEADER
I20260925 09:28:00.151298 21516 consensus_queue.cc:237] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c88127f42e748a3b0e70fc2f1d19318" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41945 } } peers { permanent_uuid: "ef1ea2b64acd41999135a613387925c9" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44751 } } peers { permanent_uuid: "a590ad2ffd9b4920973fd61dab247917" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38735 } }
I20260925 09:28:00.155481 21035 catalog_manager.cc:5495] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c88127f42e748a3b0e70fc2f1d19318 (127.10.114.66). New cstate: current_term: 1 leader_uuid: "4c88127f42e748a3b0e70fc2f1d19318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c88127f42e748a3b0e70fc2f1d19318" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41945 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef1ea2b64acd41999135a613387925c9" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a590ad2ffd9b4920973fd61dab247917" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38735 } health_report { overall_health: UNKNOWN } } }
W20260925 09:28:00.187167 21493 tablet.cc:2369] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:28:00.203574 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.198604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53484 (local address 127.10.114.66:41945)
0925 09:28:00.198796 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:00.198800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:00.198814 (+ 14us) server_negotiation.cc:407] Connection header received
0925 09:28:00.198890 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:00.198894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:00.198963 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:00.199080 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:28:00.199385 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.200137 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.201399 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.201799 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.201888 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:00.202334 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:28:00.202936 (+ 602us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:28:00.203085 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:00.203201 (+ 116us) server_negotiation.cc:299] Negotiation successful
0925 09:28:00.203279 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260925 09:28:00.206194 21360 tablet.cc:2369] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:28:00.211514 21447 raft_consensus.cc:1261] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917 [term 1 FOLLOWER]: Refusing update from remote peer 4c88127f42e748a3b0e70fc2f1d19318: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:28:00.211558 21180 raft_consensus.cc:1261] T 995b5361e422434491159ba09293a901 P ef1ea2b64acd41999135a613387925c9 [term 1 FOLLOWER]: Refusing update from remote peer 4c88127f42e748a3b0e70fc2f1d19318: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:28:00.212423 21521 consensus_queue.cc:1035] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a590ad2ffd9b4920973fd61dab247917" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 38735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:28:00.212751 21516 consensus_queue.cc:1035] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef1ea2b64acd41999135a613387925c9" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:28:00.215847 21226 tablet.cc:2369] T 995b5361e422434491159ba09293a901 P ef1ea2b64acd41999135a613387925c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:28:00.224125 21524 mvcc.cc:205] Tried to move back new op lower bound from 7333185454937706496 to 7333185454703378432. Current Snapshot: MvccSnapshot[applied={T|T < 7333185454937706496}]
I20260925 09:28:00.238089 21036 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:28:00.238371 21036 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:28:00.242118 21036 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6475b02932c5465b8693fecc3b03b895: Sending DeleteTablet for 3 replicas of tablet 995b5361e422434491159ba09293a901
I20260925 09:28:00.243209 21294 tablet_service.cc:1514] Processing DeleteTablet for tablet 995b5361e422434491159ba09293a901 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:28:00 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:53470
I20260925 09:28:00.243599 21427 tablet_service.cc:1514] Processing DeleteTablet for tablet 995b5361e422434491159ba09293a901 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:28:00 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:48212
I20260925 09:28:00.243870 21160 tablet_service.cc:1514] Processing DeleteTablet for tablet 995b5361e422434491159ba09293a901 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:28:00 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:52062
I20260925 09:28:00.244338 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 21097
I20260925 09:28:00.267102 21536 tablet_replica.cc:330] T 995b5361e422434491159ba09293a901 P ef1ea2b64acd41999135a613387925c9: stopping tablet replica
I20260925 09:28:00.267666 21536 raft_consensus.cc:2229] T 995b5361e422434491159ba09293a901 P ef1ea2b64acd41999135a613387925c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:28:00.268363 21536 raft_consensus.cc:2258] T 995b5361e422434491159ba09293a901 P ef1ea2b64acd41999135a613387925c9 [term 1 FOLLOWER]: Raft consensus is shut down!
Sep 25 09:28:00 dist-test-slave-9pj8 krb5kdc[20991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:28:00.270579 21536 ts_tablet_manager.cc:1905] T 995b5361e422434491159ba09293a901 P ef1ea2b64acd41999135a613387925c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:28:00.273142 21534 tablet_replica.cc:330] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318: stopping tablet replica
I20260925 09:28:00.276844 21534 raft_consensus.cc:2229] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:28:00.277648 21534 raft_consensus.cc:2258] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:28:00.277799 21506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.245832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52068 (local address 127.10.114.65:44751)
0925 09:28:00.246026 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:00.246035 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:00.258121 (+ 12086us) server_negotiation.cc:407] Connection header received
0925 09:28:00.258597 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:00.258604 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:00.258698 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:00.258775 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:28:00.260812 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.262141 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.264436 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.265233 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.265340 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:00.270731 (+ 5391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:28:00.270747 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:28:00.270768 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:28:00.270840 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:28:00.274567 (+ 3727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:00.275374 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:00.275382 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:00.275385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:00.275527 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:00.275938 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:00.275944 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:00.275949 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:00.276379 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:28:00.276651 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:00.277224 (+ 573us) server_negotiation.cc:299] Negotiation successful
0925 09:28:00.277433 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:28:00.279089 21220 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:28:00.281018 21534 ts_tablet_manager.cc:1905] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:28:00.281255 21535 tablet_replica.cc:330] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917: stopping tablet replica
I20260925 09:28:00.282019 21535 raft_consensus.cc:2229] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:28:00.284245 21535 raft_consensus.cc:2258] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:28:00.286330 21535 ts_tablet_manager.cc:1905] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:28:00.289433 21534 ts_tablet_manager.cc:1918] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:28:00.289665 21534 log.cc:1196] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/995b5361e422434491159ba09293a901
I20260925 09:28:00.290251 21534 ts_tablet_manager.cc:1939] T 995b5361e422434491159ba09293a901 P 4c88127f42e748a3b0e70fc2f1d19318: Deleting consensus metadata
I20260925 09:28:00.292186 21021 catalog_manager.cc:4841] TS 4c88127f42e748a3b0e70fc2f1d19318 (127.10.114.66:41945): tablet 995b5361e422434491159ba09293a901 (table test-table [id=b2b00df78f32471fa841572165eb1636]) successfully deleted
I20260925 09:28:00.292950 21535 ts_tablet_manager.cc:1918] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:28:00.293105 21535 log.cc:1196] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/995b5361e422434491159ba09293a901
I20260925 09:28:00.293586 21535 ts_tablet_manager.cc:1939] T 995b5361e422434491159ba09293a901 P a590ad2ffd9b4920973fd61dab247917: Deleting consensus metadata
I20260925 09:28:00.295243 21023 catalog_manager.cc:4841] TS a590ad2ffd9b4920973fd61dab247917 (127.10.114.67:38735): tablet 995b5361e422434491159ba09293a901 (table test-table [id=b2b00df78f32471fa841572165eb1636]) successfully deleted
I20260925 09:28:00.367754 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 21097
W20260925 09:28:00.387408 21023 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.114.65:44751 (error 108)
I20260925 09:28:00.387612 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 21231
I20260925 09:28:00.390115 21508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.389621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:44751 (local address 127.0.0.1:52072)
0925 09:28:00.389846 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:00.389957 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:44751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260925 09:28:00.390483 21023 catalog_manager.cc:4584] TS ef1ea2b64acd41999135a613387925c9 (127.10.114.65:44751): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 995b5361e422434491159ba09293a901: Network error: Client connection negotiation failed: client connection to 127.10.114.65:44751: connect: Connection refused (error 111)
Sep 25 09:28:00 dist-test-slave-9pj8 krb5kdc[20991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.66@KRBTEST.COM
I20260925 09:28:00.407617 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.389051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53492 (local address 127.10.114.66:41945)
0925 09:28:00.389237 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:00.389267 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:00.389286 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:28:00.389357 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:00.389362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:00.389436 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:00.389534 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:28:00.391130 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.392594 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.394312 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.394985 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.395109 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:00.400278 (+ 5169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:28:00.400291 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:28:00.400307 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:28:00.400349 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:28:00.404144 (+ 3795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:00.404926 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:00.404934 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:00.404937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:00.405080 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:00.405463 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:00.405467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:00.405470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:00.405877 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:28:00.406136 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:00.407130 (+ 994us) server_negotiation.cc:299] Negotiation successful
0925 09:28:00.407327 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 09:28:00.408372 21354 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:28:00.443981 21508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.443400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:44751 (local address 127.0.0.1:52086)
0925 09:28:00.443635 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:00.443752 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:44751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260925 09:28:00.445104 21508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.444662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:44751 (local address 127.0.0.1:52100)
0925 09:28:00.444837 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:00.444928 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:44751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260925 09:28:00.505656 21508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.505117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:44751 (local address 127.0.0.1:52108)
0925 09:28:00.505345 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:00.505448 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:44751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260925 09:28:00.506699 21508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.506303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:44751 (local address 127.0.0.1:52114)
0925 09:28:00.506473 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:00.506561 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:44751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260925 09:28:00.519601 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 21231
I20260925 09:28:00.539543 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 21364
Sep 25 09:28:00 dist-test-slave-9pj8 krb5kdc[20991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.67@KRBTEST.COM
I20260925 09:28:00.552867 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.540282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48214 (local address 127.10.114.67:38735)
0925 09:28:00.540440 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:00.540444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:00.540473 (+ 29us) server_negotiation.cc:407] Connection header received
0925 09:28:00.540661 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:00.540665 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:00.540731 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:00.540810 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:28:00.541902 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.542553 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.543718 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:00.544154 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:00.544262 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:00.548053 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:28:00.548064 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:28:00.548080 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:28:00.548107 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:28:00.550563 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:00.551112 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:00.551118 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:00.551120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:00.551241 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:00.551566 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:00.551569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:00.551571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:00.551936 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:28:00.552172 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:00.552427 (+ 255us) server_negotiation.cc:299] Negotiation successful
0925 09:28:00.552617 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 09:28:00.553433 21487 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:28:00.587327 21508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.586784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:44751 (local address 127.0.0.1:52120)
0925 09:28:00.587024 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:00.587131 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:44751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260925 09:28:00.588444 21508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.588011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:44751 (local address 127.0.0.1:52134)
0925 09:28:00.588220 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:00.588314 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:44751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260925 09:28:00.640789 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 21364
I20260925 09:28:00.657428 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 21004
I20260925 09:28:00.658619 21066 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:28:00.763036 21508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:00.762367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:44751 (local address 127.0.0.1:52150)
0925 09:28:00.762614 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:00.762729 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:44751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260925 09:28:00.763453 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 21004
2026-09-25T09:28:00Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4035 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:28:00 dist-test-slave-9pj8 krb5kdc[21545](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:28:00 dist-test-slave-9pj8 krb5kdc[21545](info): set up 2 sockets
Sep 25 09:28:00 dist-test-slave-9pj8 krb5kdc[21545](info): commencing operation
krb5kdc: starting...
W20260925 09:28:01.848724 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:28:01 dist-test-slave-9pj8 krb5kdc[21545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:28:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:28:02Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.114.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.114.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.114.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/rangeradmin_127.10.114.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.114.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/rangeradmin_127.10.114.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.114.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.114.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.114.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/rangerlookup_127.10.114.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.114.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/rangerlookup_127.10.114.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.114.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.114.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.114.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/HTTP_127.10.114.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.114.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/HTTP_127.10.114.84@KRBTEST.COM.keytab.
I20260925 09:28:02.138970 10697 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskMvTxiP/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/postgres -l logfile start
2026-09-25 09:28:04.599 UTC [21575] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 09:28:04.599 UTC [21575] LOG: listening on IPv4 address "127.10.114.84", port 56911
2026-09-25 09:28:04.603 UTC [21575] LOG: listening on Unix socket "/tmp/.s.PGSQL.56911"
2026-09-25 09:28:04.624 UTC [21578] LOG: database system was shut down at 2026-09-25 09:28:03 UTC
2026-09-25 09:28:04.628 UTC [21575] LOG: database system is ready to accept connections
I20260925 09:28:05.584151 10697 mini_postgres.cc:96] Postgres bound to 56911
2026-09-25 09:28:05.600 UTC [21586] FATAL: database "slave" does not exist
127.10.114.84:56911 - accepting connections
I20260925 09:28:05.601508 10697 mini_ranger.cc:161] Starting Ranger...
I20260925 09:28:05.625674 10697 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 09:28:06.109205 10697 mini_ranger.cc:87] Created ranger Postgres database
I20260925 09:28:06.109364 10697 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ranger-admin
2026-09-25 09:28:06,397 [I] DB FLAVOR :POSTGRES
2026-09-25 09:28:06,397 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 09:28:06,397 [I] Checking connection..
2026-09-25 09:28:06,398 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 09:28:06,648 [I] Checking connection passed.
2026-09-25 09:28:06,649 [I] --------- Verifying version history table ---------
2026-09-25 09:28:06,649 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 09:28:06,878 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 09:28:06,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 09:28:07,115 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 09:28:07,115 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 09:28:07,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 09:28:07.364 UTC [21650] WARNING: there is no transaction in progress
2026-09-25 09:28:07,370 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 09:28:07,370 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 09:28:07,606 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 09:28:07,607 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 09:28:07,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 09:28:07,854 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 09:28:08,101 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9pj8.c.gcp-upstream.internal','N') ;"
2026-09-25 09:28:08,311 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 09:28:08,554 [I] Table x_portal_user does not exist in database ranger
2026-09-25 09:28:08,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 09:28:08,797 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 09:28:08,797 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 09:28:09,033 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 09:28:09,274 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 09:28:09,275 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 09:28:10.196 UTC [21758] WARNING: there is no transaction in progress
2026-09-25 09:28:10.211 UTC [21758] WARNING: there is no transaction in progress
2026-09-25 09:28:10.227 UTC [21758] WARNING: there is no transaction in progress
2026-09-25 09:28:10.242 UTC [21758] WARNING: there is no transaction in progress
2026-09-25 09:28:10.259 UTC [21758] WARNING: there is no transaction in progress
2026-09-25 09:28:10.277 UTC [21758] WARNING: there is no transaction in progress
2026-09-25 09:28:10.327 UTC [21758] WARNING: there is no transaction in progress
2026-09-25 09:28:10.334 UTC [21758] WARNING: there is no transaction in progress
2026-09-25 09:28:10.348 UTC [21758] WARNING: there is no transaction in progress
2026-09-25 09:28:10.357 UTC [21758] WARNING: there is no transaction in progress
2026-09-25 09:28:10.367 UTC [21758] WARNING: there is no transaction in progress
2026-09-25 09:28:10.380 UTC [21758] WARNING: there is no transaction in progress
2026-09-25 09:28:10.389 UTC [21758] WARNING: there is no transaction in progress
2026-09-25 09:28:10.397 UTC [21758] WARNING: there is no transaction in progress
2026-09-25 09:28:10.404 UTC [21758] WARNING: there is no transaction in progress
2026-09-25 09:28:10.410 UTC [21758] WARNING: there is no transaction in progress
2026-09-25 09:28:11,021 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 09:28:11,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 09:28:11,248 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 09:28:11,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 09:28:11,495 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 09:28:11,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 09:28:11,749 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 09:28:11,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 09:28:11,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 09:28:12,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9pj8.c.gcp-upstream.internal';"
2026-09-25 09:28:12,440 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 09:28:12,440 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 09:28:12,671 [I] DB_PATCHES have already been applied
I20260925 09:28:12.676746 10697 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ranger-admin:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskMvTxiP/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 09:28:12.676829 10697 mini_ranger.cc:193] Using host: 127.10.114.84
I20260925 09:28:12.691270 10697 mini_ranger.cc:227] Ranger admin URL: http://127.10.114.84:45053
Sep 25, 2026 9:28:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 9:28:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 9:28:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 9:28:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 9:28:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 9:28:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 9:28:13 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 09:28:13.782748 10697 mini_ranger.cc:160] Time spent starting Ranger: real 8.181s user 0.000s sys 0.035s
Sep 25, 2026 9:28:32 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 9:28:32 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 25, 2026 9:28:32 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 9:28:32 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 9:28:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 9:28:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 9:28:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 9:28:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 9:28:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 9:28:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 9:28:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 9:28:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 9:28:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 9:28:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 9:28:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 9:28:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 9:28:32 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 9:28:33 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 09:28:33.771217 10697 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.114.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.114.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.114.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/rangerkms_127.10.114.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.114.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/rangerkms_127.10.114.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.114.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.114.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.114.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/HTTP_127.10.114.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.114.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/HTTP_127.10.114.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 09:28:33 dist-test-slave-9pj8 krb5kdc[21545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328513, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 09:28:33.932081 10697 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 09:28:33.951849 10697 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 09:28:34.793977 10697 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 09:28:34.794166 10697 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ranger-kms
I20260925 09:28:34.794286 10697 mini_ranger_kms.cc:227] Using postgres at 127.10.114.84:56911
2026-09-25 09:28:35,195 [I] DB FLAVOR :POSTGRES
2026-09-25 09:28:35,195 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 09:28:35,195 [I] Checking connection
2026-09-25 09:28:35,195 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 09:28:35,453 [I] connection success
2026-09-25 09:28:35,453 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 09:28:35,453 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 09:28:35,453 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 09:28:35,740 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 09:28:35,740 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 09:28:35,741 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 09:28:35,741 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:56911/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 09:28:35,985 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 09:28:36.430229 10697 mini_ranger_kms.cc:310] Created kms service
I20260925 09:28:36.627707 10697 mini_ranger_kms.cc:326] Created kudu user
I20260925 09:28:36.668220 10697 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 09:28:36 dist-test-slave-9pj8 krb5kdc[21545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328513, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
I20260925 09:28:37.023900 10697 mini_ranger_kms.cc:384] Added ranger policy
I20260925 09:28:37.024245 10697 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ranger-kms:/tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 09:28:37.024314 10697 mini_ranger_kms.cc:240] Using host: 127.10.114.84
I20260925 09:28:37.044950 10697 mini_ranger_kms.cc:276] Ranger KMS PID: 21938
I20260925 09:28:37.045110 10697 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.114.84:39761
Sep 25, 2026 9:28:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 9:28:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 9:28:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 9:28:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 9:28:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 9:28:38 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260925 09:28:39.314931 10697 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.383s user 0.107s sys 0.311s
I20260925 09:28:39.315203 10697 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 09:28:39.315426 10697 mini_ranger_kms.cc:401] 127.10.114.84:39761/kms/v1/keys
Sep 25 09:28:39 dist-test-slave-9pj8 krb5kdc[21545](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790328519, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.114.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 25 09:28:39 dist-test-slave-9pj8 krb5kdc[21545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328519, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.114.84@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
Sep 25 09:28:40 dist-test-slave-9pj8 krb5kdc[21545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328519, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.114.84@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
Sep 25 09:28:42 dist-test-slave-9pj8 krb5kdc[21545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:28:42.405357 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:33297
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:37477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:33297
--ranger_config_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.114.84:39761/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:28:42.745635 21989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:28:42.745939 21989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:28:42.746099 21989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:28:42.752842 21989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:28:42.752985 21989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:28:42.753096 21989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:28:42.753211 21989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:28:42.753307 21989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:28:42.763015 21989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:37477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.114.84:39761/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:33297
--ranger_config_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:33297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:28:42.764211 21989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:28:42.765417 21989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:28:42.777208 21995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:28:42.777254 21994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:28:42.777910 21997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:28:42.777957 21989 server_base.cc:1034] running on GCE node
I20260925 09:28:42.778720 21989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:28:42.780432 21989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:28:42.781661 21989 hybrid_clock.cc:648] HybridClock initialized: now 1790328522781619 us; error 39 us; skew 500 ppm
I20260925 09:28:42.781982 21989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:28:42 dist-test-slave-9pj8 krb5kdc[21545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328522, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:28:42.785030 21989 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:28:42.786254 21989 webserver.cc:460] Webserver started at http://127.10.114.126:41777/ using document root <none> and password file <none>
I20260925 09:28:42.786778 21989 fs_manager.cc:359] Metadata directory not provided
I20260925 09:28:42.786864 21989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:28:42.787084 21989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 09:28:42 dist-test-slave-9pj8 krb5kdc[21545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328522, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
I20260925 09:28:42.845950 21989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "7fbc166847564cb48c06286bfba578f3"
format_stamp: "Formatted at 2026-09-25 09:28:42 on dist-test-slave-9pj8"
server_key: "ec9d84783074505ba13059858743032a"
server_key_iv: "2d97a9b3a31cb88f1751080848ad0e0a"
server_key_version: "kuduclusterkey@0"
I20260925 09:28:42.846680 21989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "7fbc166847564cb48c06286bfba578f3"
format_stamp: "Formatted at 2026-09-25 09:28:42 on dist-test-slave-9pj8"
server_key: "ec9d84783074505ba13059858743032a"
server_key_iv: "2d97a9b3a31cb88f1751080848ad0e0a"
server_key_version: "kuduclusterkey@0"
I20260925 09:28:42.851761 21989 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260925 09:28:42.868494 22004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:28:42.870187 21989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:28:42.870350 21989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "7fbc166847564cb48c06286bfba578f3"
format_stamp: "Formatted at 2026-09-25 09:28:42 on dist-test-slave-9pj8"
server_key: "ec9d84783074505ba13059858743032a"
server_key_iv: "2d97a9b3a31cb88f1751080848ad0e0a"
server_key_version: "kuduclusterkey@0"
I20260925 09:28:42.870555 21989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:28:42.891294 21989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:28:42.894740 21989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:28:42.895030 21989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:28:42.913516 21989 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:33297
I20260925 09:28:42.913626 22055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:33297 every 8 connection(s)
I20260925 09:28:42.914752 21989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:28:42.918503 22056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:28:42.921717 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 21989
I20260925 09:28:42.921933 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:28:42.922562 10697 external_mini_cluster.cc:1428] Setting key c6b7ae521a5e7a718b1a73afad692900
I20260925 09:28:42.932397 22056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3: Bootstrap starting.
I20260925 09:28:42.936239 22056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3: Neither blocks nor log segments found. Creating new log.
I20260925 09:28:42.937211 22056 log.cc:824] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3: Log is configured to *not* fsync() on all Append() calls
Sep 25 09:28:42 dist-test-slave-9pj8 krb5kdc[21545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:28:42.941622 22056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3: No bootstrap required, opened a new log
I20260925 09:28:42.947237 22059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:42.924536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60460 (local address 127.10.114.126:33297)
0925 09:28:42.925361 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:42.925379 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:42.925428 (+ 49us) server_negotiation.cc:407] Connection header received
0925 09:28:42.927940 (+ 2512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:42.927968 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:42.928503 (+ 535us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:42.928960 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:28:42.930200 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:42.932272 (+ 2072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:42.933517 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:42.934669 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:42.934876 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:42.939066 (+ 4190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:28:42.939075 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:28:42.939100 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:28:42.939132 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:28:42.943160 (+ 4028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:42.943807 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:42.943815 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:42.943828 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:42.944021 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:42.944355 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:42.944361 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:42.944365 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:42.945019 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:28:42.945294 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:42.945612 (+ 318us) server_negotiation.cc:299] Negotiation successful
0925 09:28:42.945967 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":337,"threads_started":1}
I20260925 09:28:42.950879 22056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fbc166847564cb48c06286bfba578f3" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33297 } }
I20260925 09:28:42.951248 22056 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:28:42.951345 22056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fbc166847564cb48c06286bfba578f3, State: Initialized, Role: FOLLOWER
I20260925 09:28:42.952055 22056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fbc166847564cb48c06286bfba578f3" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33297 } }
I20260925 09:28:42.952312 22056 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:28:42.952464 22056 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:28:42.952630 22056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:28:42.954543 22056 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fbc166847564cb48c06286bfba578f3" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33297 } }
I20260925 09:28:42.954960 22056 leader_election.cc:302] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fbc166847564cb48c06286bfba578f3; no voters:
I20260925 09:28:42.955530 22056 leader_election.cc:288] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:28:42.955657 22061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:28:42.956707 22061 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3 [term 1 LEADER]: Becoming Leader. State: Replica: 7fbc166847564cb48c06286bfba578f3, State: Running, Role: LEADER
I20260925 09:28:42.957176 22061 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fbc166847564cb48c06286bfba578f3" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33297 } }
I20260925 09:28:42.957670 22056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:28:42.959602 22062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7fbc166847564cb48c06286bfba578f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fbc166847564cb48c06286bfba578f3" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33297 } } }
I20260925 09:28:42.959916 22062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3 [sys.catalog]: This master's current role is: LEADER
I20260925 09:28:42.960227 22063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fbc166847564cb48c06286bfba578f3. Latest consensus state: current_term: 1 leader_uuid: "7fbc166847564cb48c06286bfba578f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fbc166847564cb48c06286bfba578f3" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33297 } } }
I20260925 09:28:42.960530 22063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3 [sys.catalog]: This master's current role is: LEADER
I20260925 09:28:42.962543 22066 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:28:42.967816 22066 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:28:42.973934 22056 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 09:28:42.982811 22066 catalog_manager.cc:1347] Generated new cluster ID: 1c4ac1c90ce24027aa4cdd770d3d886f
I20260925 09:28:42.983114 22066 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:28:42.998366 22066 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:28:42.999681 22066 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:28:43.012928 22066 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3: Generated new TSK 0
I20260925 09:28:43.013777 22066 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
I20260925 09:28:43.796622 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.114.84:39761/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:33297
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:37477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:28:44 dist-test-slave-9pj8 krb5kdc[21545](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790328524, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260925 09:28:44.236517 22101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:28:44.236905 22101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:28:44.237079 22101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:28:44.248860 22101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:28:44.248999 22101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:28:44.249203 22101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:28:44.264802 22101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:37477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.114.84:39761/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:33297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:28:44.266472 22101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:28:44.268187 22101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:28:44.284446 22111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:28:44.285782 22112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:28:44.294194 22101 server_base.cc:1034] running on GCE node
W20260925 09:28:44.296547 22114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:28:44.297454 22101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:28:44.298646 22101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:28:44.299896 22101 hybrid_clock.cc:648] HybridClock initialized: now 1790328524299829 us; error 54 us; skew 500 ppm
I20260925 09:28:44.300328 22101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:28:44 dist-test-slave-9pj8 krb5kdc[21545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328524, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:28:44.304319 22101 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:28:44.306041 22101 webserver.cc:460] Webserver started at http://127.10.114.65:43287/ using document root <none> and password file <none>
I20260925 09:28:44.306751 22101 fs_manager.cc:359] Metadata directory not provided
I20260925 09:28:44.306890 22101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:28:44.307214 22101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 09:28:44 dist-test-slave-9pj8 krb5kdc[21545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328524, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
I20260925 09:28:44.324819 22101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "32bdcc62fe7c475ab6c79751686aea71"
format_stamp: "Formatted at 2026-09-25 09:28:44 on dist-test-slave-9pj8"
server_key: "00d25064baab5159b0679d9f127b9fae"
server_key_iv: "a77b6da01e76f617160dc6e4d6f3f7f3"
server_key_version: "kuduclusterkey@0"
I20260925 09:28:44.325657 22101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "32bdcc62fe7c475ab6c79751686aea71"
format_stamp: "Formatted at 2026-09-25 09:28:44 on dist-test-slave-9pj8"
server_key: "00d25064baab5159b0679d9f127b9fae"
server_key_iv: "a77b6da01e76f617160dc6e4d6f3f7f3"
server_key_version: "kuduclusterkey@0"
I20260925 09:28:44.332005 22101 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260925 09:28:44.357448 22121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:28:44.362653 22101 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.002s sys 0.000s
I20260925 09:28:44.362852 22101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "32bdcc62fe7c475ab6c79751686aea71"
format_stamp: "Formatted at 2026-09-25 09:28:44 on dist-test-slave-9pj8"
server_key: "00d25064baab5159b0679d9f127b9fae"
server_key_iv: "a77b6da01e76f617160dc6e4d6f3f7f3"
server_key_version: "kuduclusterkey@0"
I20260925 09:28:44.363101 22101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:28:44.390606 22101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:28:44.394583 22101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:28:44.394906 22101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:28:44.395893 22101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:28:44.397260 22101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:28:44.397359 22101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:28:44.397500 22101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:28:44.397568 22101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:28:44.476147 22101 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:44427
I20260925 09:28:44.476157 22233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:44427 every 8 connection(s)
I20260925 09:28:44.477708 22101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:28:44.478888 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 22101
I20260925 09:28:44.479143 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:28:44.480195 10697 external_mini_cluster.cc:1428] Setting key 2af87a4e90817b739a4db7b53851b584
WARNING: no policy specified for kudu/127.10.114.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.66@KRBTEST.COM" created.
Sep 25 09:28:44 dist-test-slave-9pj8 krb5kdc[21545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328524, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:28:44.539613 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:44.487398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:33297 (local address 127.10.114.65:47659)
0925 09:28:44.488739 (+ 1341us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:44.488811 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0925 09:28:44.492404 (+ 3593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:28:44.508254 (+ 15850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:28:44.508273 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:28:44.509172 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:28:44.510273 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:44.510304 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:44.512517 (+ 2213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:44.512526 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:44.515408 (+ 2882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:44.515429 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:44.519119 (+ 3690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:44.519130 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:44.519643 (+ 513us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:44.528029 (+ 8386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:28:44.528080 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:28:44.531180 (+ 3100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:28:44.535181 (+ 4001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:28:44.535189 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:28:44.535207 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:28:44.535701 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:28:44.536136 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:28:44.536141 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:28:44.536144 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:28:44.536374 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:28:44.537008 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:28:44.537015 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:28:44.537427 (+ 412us) client_negotiation.cc:769] Sending connection context
0925 09:28:44.537711 (+ 284us) client_negotiation.cc:240] Negotiation successful
0925 09:28:44.538248 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":917,"thread_start_us":600,"threads_started":1}
I20260925 09:28:44.540647 22237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:44.487957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:47659 (local address 127.10.114.126:33297)
0925 09:28:44.507257 (+ 19300us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:44.507267 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:44.507294 (+ 27us) server_negotiation.cc:407] Connection header received
0925 09:28:44.507392 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:44.507401 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:44.507527 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:44.510521 (+ 2994us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:28:44.510759 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:44.512276 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:44.515664 (+ 3388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:44.516604 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:44.516739 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:44.531521 (+ 14782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:28:44.531529 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:28:44.531534 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:28:44.531574 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:28:44.535019 (+ 3445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:44.535844 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:44.535849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:44.535853 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:44.536001 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:44.536495 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:44.536499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:44.536502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:44.536813 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:28:44.536923 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:44.540134 (+ 3211us) server_negotiation.cc:299] Negotiation successful
0925 09:28:44.540341 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19090,"thread_start_us":19694,"threads_started":1}
I20260925 09:28:44.542053 22234 heartbeater.cc:347] Connected to a master server at 127.10.114.126:33297
I20260925 09:28:44.542388 22234 heartbeater.cc:464] Registering TS with master...
I20260925 09:28:44.543195 22234 heartbeater.cc:511] Master 127.10.114.126:33297 requested a full tablet report, sending...
I20260925 09:28:44.545501 22021 ts_manager.cc:194] Registered new tserver with Master: 32bdcc62fe7c475ab6c79751686aea71 (127.10.114.65:44427)
I20260925 09:28:44.547564 22021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:47659
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
I20260925 09:28:44.556617 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.114.84:39761/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:33297
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:37477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:28:44 dist-test-slave-9pj8 krb5kdc[21545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328524, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
Sep 25 09:28:44 dist-test-slave-9pj8 krb5kdc[21545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328524, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
W20260925 09:28:44.973219 22240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:28:44.973523 22240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:28:44.973690 22240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:28:44.984884 22240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:28:44.985069 22240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:28:44.985299 22240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:28:44.999094 22240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:37477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.114.84:39761/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:33297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:28:45.000821 22240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:28:45.002260 22240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:28:45.023849 22248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:28:45.025444 22247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:28:45.029512 22250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:28:45.030028 22240 server_base.cc:1034] running on GCE node
I20260925 09:28:45.030843 22240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:28:45.032543 22240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:28:45.033845 22240 hybrid_clock.cc:648] HybridClock initialized: now 1790328525033785 us; error 45 us; skew 500 ppm
I20260925 09:28:45.034221 22240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:28:45 dist-test-slave-9pj8 krb5kdc[21545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328525, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:28:45.038415 22240 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:28:45.088253 22240 webserver.cc:460] Webserver started at http://127.10.114.66:33341/ using document root <none> and password file <none>
I20260925 09:28:45.089411 22240 fs_manager.cc:359] Metadata directory not provided
I20260925 09:28:45.089635 22240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:28:45.090051 22240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 09:28:45 dist-test-slave-9pj8 krb5kdc[21545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328525, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
I20260925 09:28:45.114863 22240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "f4a37fc526654e7aa1548c3971dacb35"
format_stamp: "Formatted at 2026-09-25 09:28:45 on dist-test-slave-9pj8"
server_key: "9b9d67a8a3bf74021956666eb3ef5655"
server_key_iv: "88743aeaef74219b385766a9a60f417f"
server_key_version: "kuduclusterkey@0"
I20260925 09:28:45.116250 22240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "f4a37fc526654e7aa1548c3971dacb35"
format_stamp: "Formatted at 2026-09-25 09:28:45 on dist-test-slave-9pj8"
server_key: "9b9d67a8a3bf74021956666eb3ef5655"
server_key_iv: "88743aeaef74219b385766a9a60f417f"
server_key_version: "kuduclusterkey@0"
I20260925 09:28:45.126854 22240 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.005s sys 0.004s
I20260925 09:28:45.171108 22257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:28:45.175051 22240 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.006s sys 0.004s
I20260925 09:28:45.175366 22240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "f4a37fc526654e7aa1548c3971dacb35"
format_stamp: "Formatted at 2026-09-25 09:28:45 on dist-test-slave-9pj8"
server_key: "9b9d67a8a3bf74021956666eb3ef5655"
server_key_iv: "88743aeaef74219b385766a9a60f417f"
server_key_version: "kuduclusterkey@0"
I20260925 09:28:45.175711 22240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:28:45.210965 22240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:28:45.216058 22240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:28:45.216370 22240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:28:45.217479 22240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:28:45.219273 22240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:28:45.219372 22240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:28:45.219511 22240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:28:45.219578 22240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:28:45.287037 22240 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:42971
I20260925 09:28:45.287057 22376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:42971 every 8 connection(s)
I20260925 09:28:45.288399 22240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:28:45.292565 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 22240
I20260925 09:28:45.292779 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:28:45.293334 10697 external_mini_cluster.cc:1428] Setting key b1b74d8289955e28337c4c4499c57c7f
Sep 25 09:28:45 dist-test-slave-9pj8 krb5kdc[21545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328525, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:28:45.312070 22379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:45.291423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:51647 (local address 127.10.114.126:33297)
0925 09:28:45.291927 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:45.291931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:45.293326 (+ 1395us) server_negotiation.cc:407] Connection header received
0925 09:28:45.295512 (+ 2186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:45.295517 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:45.295615 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:45.295727 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:28:45.297155 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.297867 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.300508 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.300928 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.301064 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:45.304382 (+ 3318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:28:45.304390 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:28:45.304395 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:28:45.304436 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:28:45.308200 (+ 3764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:45.309144 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:45.309151 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:45.309155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:45.309311 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:45.309851 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:45.309857 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:45.309861 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:45.310253 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:28:45.310429 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:45.311321 (+ 892us) server_negotiation.cc:299] Negotiation successful
0925 09:28:45.311583 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":252,"threads_started":1}
I20260925 09:28:45.313252 22380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:45.292257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:33297 (local address 127.10.114.66:51647)
0925 09:28:45.293147 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:45.293198 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 09:28:45.295201 (+ 2003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:28:45.296003 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:28:45.296012 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:28:45.296429 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:28:45.296993 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:45.297011 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.298025 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.298030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:45.299496 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:45.299514 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.301144 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.301151 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:45.301379 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:45.302076 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:28:45.302098 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:28:45.304141 (+ 2043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:28:45.308355 (+ 4214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:28:45.308366 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:28:45.308386 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:28:45.308990 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:28:45.309441 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:28:45.309446 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:28:45.309449 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:28:45.309684 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:28:45.310418 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:28:45.310429 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:28:45.310854 (+ 425us) client_negotiation.cc:769] Sending connection context
0925 09:28:45.311159 (+ 305us) client_negotiation.cc:240] Negotiation successful
0925 09:28:45.311720 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":332,"threads_started":1}
I20260925 09:28:45.314855 22377 heartbeater.cc:347] Connected to a master server at 127.10.114.126:33297
I20260925 09:28:45.315212 22377 heartbeater.cc:464] Registering TS with master...
I20260925 09:28:45.316099 22377 heartbeater.cc:511] Master 127.10.114.126:33297 requested a full tablet report, sending...
I20260925 09:28:45.317834 22021 ts_manager.cc:194] Registered new tserver with Master: f4a37fc526654e7aa1548c3971dacb35 (127.10.114.66:42971)
I20260925 09:28:45.319171 22021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:51647
WARNING: no policy specified for kudu/127.10.114.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
I20260925 09:28:45.347280 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.114.84:39761/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:33297
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:37477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:28:45.551259 22234 heartbeater.cc:503] Master 127.10.114.126:33297 was elected leader, sending a full tablet report...
W20260925 09:28:45.673105 22383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:28:45.673348 22383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:28:45.673491 22383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:28:45.681128 22383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:28:45.681229 22383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:28:45.681412 22383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:28:45.690634 22383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:37477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.114.84:39761/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:33297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:28:45.691877 22383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:28:45.693076 22383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:28:45.706037 22388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:28:45.706003 22391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:28:45.706072 22389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:28:45.707084 22383 server_base.cc:1034] running on GCE node
I20260925 09:28:45.707620 22383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:28:45.708518 22383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:28:45.709729 22383 hybrid_clock.cc:648] HybridClock initialized: now 1790328525709702 us; error 50 us; skew 500 ppm
I20260925 09:28:45.710019 22383 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:28:45 dist-test-slave-9pj8 krb5kdc[21545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328525, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:28:45.712955 22383 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:28:45.714558 22383 webserver.cc:460] Webserver started at http://127.10.114.67:34063/ using document root <none> and password file <none>
I20260925 09:28:45.715157 22383 fs_manager.cc:359] Metadata directory not provided
I20260925 09:28:45.715291 22383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:28:45.715574 22383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 09:28:45 dist-test-slave-9pj8 krb5kdc[21545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328525, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
I20260925 09:28:45.731091 22383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "de45e69449884b6e98fe25b6e3dc61e6"
format_stamp: "Formatted at 2026-09-25 09:28:45 on dist-test-slave-9pj8"
server_key: "8c322251260a77f27b9dc1abe542c98e"
server_key_iv: "c04d8bb2ef6815b45c01f26a0b44c0a8"
server_key_version: "kuduclusterkey@0"
I20260925 09:28:45.732185 22383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "de45e69449884b6e98fe25b6e3dc61e6"
format_stamp: "Formatted at 2026-09-25 09:28:45 on dist-test-slave-9pj8"
server_key: "8c322251260a77f27b9dc1abe542c98e"
server_key_iv: "c04d8bb2ef6815b45c01f26a0b44c0a8"
server_key_version: "kuduclusterkey@0"
I20260925 09:28:45.738803 22383 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260925 09:28:45.752296 22398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:28:45.753682 22383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:28:45.753862 22383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "de45e69449884b6e98fe25b6e3dc61e6"
format_stamp: "Formatted at 2026-09-25 09:28:45 on dist-test-slave-9pj8"
server_key: "8c322251260a77f27b9dc1abe542c98e"
server_key_iv: "c04d8bb2ef6815b45c01f26a0b44c0a8"
server_key_version: "kuduclusterkey@0"
I20260925 09:28:45.754065 22383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:28:45.778342 22383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:28:45.782353 22383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:28:45.782636 22383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:28:45.783509 22383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:28:45.784821 22383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:28:45.784920 22383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:28:45.785080 22383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:28:45.785159 22383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:28:45.819959 22383 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:42295
I20260925 09:28:45.820034 22510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:42295 every 8 connection(s)
I20260925 09:28:45.821213 22383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:28:45.828862 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 22383
I20260925 09:28:45.829047 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:28:45.829457 10697 external_mini_cluster.cc:1428] Setting key a618087b0c205dd851b7eb81cf68e3a4
Sep 25 09:28:45 dist-test-slave-9pj8 krb5kdc[21545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328525, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:28:45.841398 22513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:45.823970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:48181 (local address 127.10.114.126:33297)
0925 09:28:45.824795 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:45.824799 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:45.825565 (+ 766us) server_negotiation.cc:407] Connection header received
0925 09:28:45.827413 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:45.827422 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:45.827565 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:45.827708 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:28:45.829485 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.830547 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.832593 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.832983 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.833080 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:45.836114 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:28:45.836119 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:28:45.836129 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:28:45.836157 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:28:45.838674 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:45.839321 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:45.839325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:45.839327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:45.839437 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:45.839979 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:45.839982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:45.839984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:45.840253 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:28:45.840355 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:45.840988 (+ 633us) server_negotiation.cc:299] Negotiation successful
0925 09:28:45.841140 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":579,"threads_started":1}
I20260925 09:28:45.842842 22514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:45.824717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:33297 (local address 127.10.114.67:48181)
0925 09:28:45.825392 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:45.825441 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 09:28:45.827129 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:28:45.828030 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:28:45.828045 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:28:45.828575 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:28:45.829296 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:45.829316 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.830737 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.830741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:45.832423 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:45.832453 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.833165 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.833169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:45.833354 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:45.833981 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:28:45.834004 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:28:45.835921 (+ 1917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:28:45.838793 (+ 2872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:28:45.838799 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:28:45.838810 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:28:45.839214 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:28:45.839565 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:28:45.839569 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:28:45.839572 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:28:45.839868 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:28:45.840370 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:28:45.840376 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:28:45.840671 (+ 295us) client_negotiation.cc:769] Sending connection context
0925 09:28:45.840914 (+ 243us) client_negotiation.cc:240] Negotiation successful
0925 09:28:45.841312 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":260,"threads_started":1}
I20260925 09:28:45.844165 22511 heartbeater.cc:347] Connected to a master server at 127.10.114.126:33297
I20260925 09:28:45.844491 22511 heartbeater.cc:464] Registering TS with master...
I20260925 09:28:45.845287 22511 heartbeater.cc:511] Master 127.10.114.126:33297 requested a full tablet report, sending...
I20260925 09:28:45.846710 22021 ts_manager.cc:194] Registered new tserver with Master: de45e69449884b6e98fe25b6e3dc61e6 (127.10.114.67:42295)
I20260925 09:28:45.847503 22021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:48181
I20260925 09:28:45.856756 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:28:45.873652 22513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:45.861304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41076 (local address 127.10.114.126:33297)
0925 09:28:45.861456 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:45.861460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:45.861944 (+ 484us) server_negotiation.cc:407] Connection header received
0925 09:28:45.862066 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:45.862069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:45.862142 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:45.862216 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:28:45.863664 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.864411 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.865577 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.865975 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.866063 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:45.868545 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:28:45.868549 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:28:45.868551 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:28:45.868575 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:28:45.871391 (+ 2816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:45.872138 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:45.872141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:45.872143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:45.872255 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:45.872595 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:45.872598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:45.872600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:45.872820 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:28:45.872993 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:45.873305 (+ 312us) server_negotiation.cc:299] Negotiation successful
0925 09:28:45.873446 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 09:28:45.877627 22021 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:28:45.884963 22021 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:28:45.920398 22525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:45.896798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:42295 (local address 127.0.0.1:43792)
0925 09:28:45.908176 (+ 11378us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:45.908231 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 09:28:45.908460 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:28:45.911938 (+ 3478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:28:45.911944 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:28:45.911977 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:28:45.912423 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:45.912435 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.914779 (+ 2344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.914784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:45.917327 (+ 2543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:45.917336 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.919325 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.919331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:45.919508 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:45.919567 (+ 59us) client_negotiation.cc:769] Sending connection context
0925 09:28:45.919685 (+ 118us) client_negotiation.cc:240] Negotiation successful
0925 09:28:45.919920 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11068,"mutex_wait_us":236,"thread_start_us":14527,"threads_started":1}
I20260925 09:28:45.920662 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:45.897076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43792 (local address 127.10.114.67:42295)
0925 09:28:45.911228 (+ 14152us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:45.911237 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:45.911267 (+ 30us) server_negotiation.cc:407] Connection header received
0925 09:28:45.911358 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:45.911368 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:45.911700 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:45.912606 (+ 906us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:28:45.912834 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.914639 (+ 1805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.917467 (+ 2828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.918831 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.918949 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:45.919156 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:45.920139 (+ 983us) server_negotiation.cc:299] Negotiation successful
0925 09:28:45.920307 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14013,"thread_start_us":13852,"threads_started":1}
I20260925 09:28:45.921141 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:45.897173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46602 (local address 127.10.114.66:42971)
0925 09:28:45.910582 (+ 13409us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:45.910593 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:45.911966 (+ 1373us) server_negotiation.cc:407] Connection header received
0925 09:28:45.912072 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:45.912079 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:45.912327 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:45.912490 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:28:45.913491 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.914795 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.918070 (+ 3275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.919426 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.919524 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:45.919686 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:45.920366 (+ 680us) server_negotiation.cc:299] Negotiation successful
0925 09:28:45.920803 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13219,"thread_start_us":13427,"threads_started":1}
I20260925 09:28:45.921591 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:45.909103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59420 (local address 127.10.114.65:44427)
0925 09:28:45.911304 (+ 2201us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:45.911314 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:45.912096 (+ 782us) server_negotiation.cc:407] Connection header received
0925 09:28:45.912383 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:45.912390 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:45.912583 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:45.912744 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:28:45.913848 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.915790 (+ 1942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.918057 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.919418 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.919517 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:45.919729 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:45.921130 (+ 1401us) server_negotiation.cc:299] Negotiation successful
0925 09:28:45.921257 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2034,"thread_start_us":1885,"threads_started":1}
I20260925 09:28:45.922241 22529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:45.911262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:44427 (local address 127.0.0.1:59420)
0925 09:28:45.911978 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:45.911995 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:28:45.912191 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:28:45.913047 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:28:45.913052 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:28:45.913105 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:28:45.913669 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:45.913681 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.915971 (+ 2290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.915976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:45.917892 (+ 1916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:45.917905 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.919938 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.919944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:45.920920 (+ 976us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:45.920957 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 09:28:45.921687 (+ 730us) client_negotiation.cc:240] Negotiation successful
0925 09:28:45.921812 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":826,"thread_start_us":423,"threads_started":1}
I20260925 09:28:45.922713 22524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:45.897013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:42971 (local address 127.0.0.1:46602)
0925 09:28:45.911659 (+ 14646us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:45.911681 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:28:45.911834 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:28:45.912757 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:28:45.912761 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:28:45.912781 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:28:45.913261 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:45.913273 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.914950 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.914955 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:45.917160 (+ 2205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:45.917172 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:45.919885 (+ 2713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:45.919890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:45.920138 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:45.920170 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 09:28:45.921811 (+ 1641us) client_negotiation.cc:240] Negotiation successful
0925 09:28:45.921923 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14509,"mutex_wait_us":52,"thread_start_us":10695,"threads_started":1}
I20260925 09:28:45.924574 22446 tablet_service.cc:1467] Processing CreateTablet for tablet f00ad05feefb4922b848d18ac06e73ce (DEFAULT_TABLE table=test-table [id=1031ec61258e411f9cf41ae8ae51a781]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:28:45.926088 22446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f00ad05feefb4922b848d18ac06e73ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:28:45.925788 22169 tablet_service.cc:1467] Processing CreateTablet for tablet f00ad05feefb4922b848d18ac06e73ce (DEFAULT_TABLE table=test-table [id=1031ec61258e411f9cf41ae8ae51a781]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:28:45.926085 22312 tablet_service.cc:1467] Processing CreateTablet for tablet f00ad05feefb4922b848d18ac06e73ce (DEFAULT_TABLE table=test-table [id=1031ec61258e411f9cf41ae8ae51a781]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:28:45.926980 22169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f00ad05feefb4922b848d18ac06e73ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:28:45.927294 22312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f00ad05feefb4922b848d18ac06e73ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:28:45.941243 22531 tablet_bootstrap.cc:492] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6: Bootstrap starting.
I20260925 09:28:45.942977 22530 tablet_bootstrap.cc:492] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71: Bootstrap starting.
I20260925 09:28:45.944461 22532 tablet_bootstrap.cc:492] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35: Bootstrap starting.
I20260925 09:28:45.945024 22531 tablet_bootstrap.cc:654] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6: Neither blocks nor log segments found. Creating new log.
I20260925 09:28:45.945993 22531 log.cc:824] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6: Log is configured to *not* fsync() on all Append() calls
I20260925 09:28:45.946416 22530 tablet_bootstrap.cc:654] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71: Neither blocks nor log segments found. Creating new log.
I20260925 09:28:45.948050 22530 log.cc:824] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71: Log is configured to *not* fsync() on all Append() calls
I20260925 09:28:45.948307 22532 tablet_bootstrap.cc:654] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35: Neither blocks nor log segments found. Creating new log.
I20260925 09:28:45.949275 22532 log.cc:824] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35: Log is configured to *not* fsync() on all Append() calls
I20260925 09:28:45.950147 22531 tablet_bootstrap.cc:492] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6: No bootstrap required, opened a new log
I20260925 09:28:45.950398 22531 ts_tablet_manager.cc:1397] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.007s
I20260925 09:28:45.951637 22530 tablet_bootstrap.cc:492] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71: No bootstrap required, opened a new log
I20260925 09:28:45.951918 22530 ts_tablet_manager.cc:1397] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260925 09:28:45.952615 22532 tablet_bootstrap.cc:492] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35: No bootstrap required, opened a new log
I20260925 09:28:45.952845 22532 ts_tablet_manager.cc:1397] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260925 09:28:45.959830 22530 raft_consensus.cc:348] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de45e69449884b6e98fe25b6e3dc61e6" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 42295 } } peers { permanent_uuid: "f4a37fc526654e7aa1548c3971dacb35" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42971 } } peers { permanent_uuid: "32bdcc62fe7c475ab6c79751686aea71" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44427 } }
I20260925 09:28:45.960165 22530 raft_consensus.cc:374] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:28:45.960265 22530 raft_consensus.cc:729] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32bdcc62fe7c475ab6c79751686aea71, State: Initialized, Role: FOLLOWER
I20260925 09:28:45.960831 22530 consensus_queue.cc:260] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de45e69449884b6e98fe25b6e3dc61e6" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 42295 } } peers { permanent_uuid: "f4a37fc526654e7aa1548c3971dacb35" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42971 } } peers { permanent_uuid: "32bdcc62fe7c475ab6c79751686aea71" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44427 } }
I20260925 09:28:45.961112 22531 raft_consensus.cc:348] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de45e69449884b6e98fe25b6e3dc61e6" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 42295 } } peers { permanent_uuid: "f4a37fc526654e7aa1548c3971dacb35" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42971 } } peers { permanent_uuid: "32bdcc62fe7c475ab6c79751686aea71" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44427 } }
I20260925 09:28:45.961571 22531 raft_consensus.cc:374] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:28:45.961696 22531 raft_consensus.cc:729] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de45e69449884b6e98fe25b6e3dc61e6, State: Initialized, Role: FOLLOWER
I20260925 09:28:45.962448 22530 ts_tablet_manager.cc:1428] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260925 09:28:45.962373 22531 consensus_queue.cc:260] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de45e69449884b6e98fe25b6e3dc61e6" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 42295 } } peers { permanent_uuid: "f4a37fc526654e7aa1548c3971dacb35" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42971 } } peers { permanent_uuid: "32bdcc62fe7c475ab6c79751686aea71" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44427 } }
I20260925 09:28:45.963400 22532 raft_consensus.cc:348] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de45e69449884b6e98fe25b6e3dc61e6" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 42295 } } peers { permanent_uuid: "f4a37fc526654e7aa1548c3971dacb35" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42971 } } peers { permanent_uuid: "32bdcc62fe7c475ab6c79751686aea71" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44427 } }
I20260925 09:28:45.963842 22532 raft_consensus.cc:374] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:28:45.963888 22511 heartbeater.cc:503] Master 127.10.114.126:33297 was elected leader, sending a full tablet report...
I20260925 09:28:45.963966 22532 raft_consensus.cc:729] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4a37fc526654e7aa1548c3971dacb35, State: Initialized, Role: FOLLOWER
I20260925 09:28:45.964567 22531 ts_tablet_manager.cc:1428] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6: Time spent starting tablet: real 0.014s user 0.007s sys 0.006s
I20260925 09:28:45.964567 22532 consensus_queue.cc:260] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de45e69449884b6e98fe25b6e3dc61e6" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 42295 } } peers { permanent_uuid: "f4a37fc526654e7aa1548c3971dacb35" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42971 } } peers { permanent_uuid: "32bdcc62fe7c475ab6c79751686aea71" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44427 } }
I20260925 09:28:45.965849 22377 heartbeater.cc:503] Master 127.10.114.126:33297 was elected leader, sending a full tablet report...
I20260925 09:28:45.967069 22532 ts_tablet_manager.cc:1428] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
I20260925 09:28:46.007916 22538 raft_consensus.cc:482] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:28:46.008198 22538 raft_consensus.cc:504] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de45e69449884b6e98fe25b6e3dc61e6" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 42295 } } peers { permanent_uuid: "f4a37fc526654e7aa1548c3971dacb35" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42971 } } peers { permanent_uuid: "32bdcc62fe7c475ab6c79751686aea71" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44427 } }
I20260925 09:28:46.009768 22538 leader_election.cc:288] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de45e69449884b6e98fe25b6e3dc61e6 (127.10.114.67:42295), 32bdcc62fe7c475ab6c79751686aea71 (127.10.114.65:44427)
I20260925 09:28:46.017086 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:46.010320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:57011 (local address 127.10.114.65:44427)
0925 09:28:46.010610 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:46.010619 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:46.011275 (+ 656us) server_negotiation.cc:407] Connection header received
0925 09:28:46.011535 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:46.011540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:46.011612 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:46.011727 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:28:46.012826 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:46.013612 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:46.015306 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:46.016155 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:46.016271 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:46.016356 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:46.016709 (+ 353us) server_negotiation.cc:299] Negotiation successful
0925 09:28:46.016831 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260925 09:28:46.017179 22539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:46.010074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:44427 (local address 127.10.114.66:57011)
0925 09:28:46.011104 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:46.011120 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:28:46.011276 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:28:46.011835 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:28:46.011843 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:28:46.011871 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:28:46.012665 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:46.012681 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:46.013757 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:46.013760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:46.015156 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:46.015169 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:46.016357 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:46.016363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:46.016555 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:46.016578 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 09:28:46.016683 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 09:28:46.016802 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":914,"mutex_wait_us":2,"thread_start_us":722,"threads_started":1}
I20260925 09:28:46.017213 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:46.010116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:52299 (local address 127.10.114.67:42295)
0925 09:28:46.010388 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:46.010397 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:46.011075 (+ 678us) server_negotiation.cc:407] Connection header received
0925 09:28:46.011251 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:46.011258 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:46.011347 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:46.011437 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:28:46.012588 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:46.013352 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:46.015172 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:46.016251 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:46.016401 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:46.016474 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:46.016847 (+ 373us) server_negotiation.cc:299] Negotiation successful
0925 09:28:46.016968 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260925 09:28:46.017906 22540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:46.010067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:42295 (local address 127.10.114.66:52299)
0925 09:28:46.010970 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0925 09:28:46.010989 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:28:46.011103 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:28:46.011563 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:28:46.011567 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:28:46.011590 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:28:46.012035 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:46.012046 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:46.013488 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:46.013493 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:46.015015 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:28:46.015026 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:28:46.016501 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:28:46.016506 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:28:46.016654 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:46.016691 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 09:28:46.017425 (+ 734us) client_negotiation.cc:240] Negotiation successful
0925 09:28:46.017556 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"thread_start_us":514,"threads_started":1}
I20260925 09:28:46.018558 22189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f00ad05feefb4922b848d18ac06e73ce" candidate_uuid: "f4a37fc526654e7aa1548c3971dacb35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32bdcc62fe7c475ab6c79751686aea71" is_pre_election: true
I20260925 09:28:46.018762 22466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f00ad05feefb4922b848d18ac06e73ce" candidate_uuid: "f4a37fc526654e7aa1548c3971dacb35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de45e69449884b6e98fe25b6e3dc61e6" is_pre_election: true
I20260925 09:28:46.019126 22189 raft_consensus.cc:2454] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4a37fc526654e7aa1548c3971dacb35 in term 0.
I20260925 09:28:46.019256 22466 raft_consensus.cc:2454] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4a37fc526654e7aa1548c3971dacb35 in term 0.
I20260925 09:28:46.019858 22259 leader_election.cc:302] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32bdcc62fe7c475ab6c79751686aea71, f4a37fc526654e7aa1548c3971dacb35; no voters:
I20260925 09:28:46.020265 22538 raft_consensus.cc:2781] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:28:46.020417 22538 raft_consensus.cc:482] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:28:46.020532 22538 raft_consensus.cc:3037] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:28:46.022634 22538 raft_consensus.cc:504] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de45e69449884b6e98fe25b6e3dc61e6" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 42295 } } peers { permanent_uuid: "f4a37fc526654e7aa1548c3971dacb35" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42971 } } peers { permanent_uuid: "32bdcc62fe7c475ab6c79751686aea71" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44427 } }
I20260925 09:28:46.023569 22538 leader_election.cc:288] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [CANDIDATE]: Term 1 election: Requested vote from peers de45e69449884b6e98fe25b6e3dc61e6 (127.10.114.67:42295), 32bdcc62fe7c475ab6c79751686aea71 (127.10.114.65:44427)
I20260925 09:28:46.024075 22466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f00ad05feefb4922b848d18ac06e73ce" candidate_uuid: "f4a37fc526654e7aa1548c3971dacb35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de45e69449884b6e98fe25b6e3dc61e6"
I20260925 09:28:46.024076 22189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f00ad05feefb4922b848d18ac06e73ce" candidate_uuid: "f4a37fc526654e7aa1548c3971dacb35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32bdcc62fe7c475ab6c79751686aea71"
I20260925 09:28:46.024282 22466 raft_consensus.cc:3037] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:28:46.024282 22189 raft_consensus.cc:3037] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:28:46.027308 22189 raft_consensus.cc:2454] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4a37fc526654e7aa1548c3971dacb35 in term 1.
I20260925 09:28:46.027308 22466 raft_consensus.cc:2454] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4a37fc526654e7aa1548c3971dacb35 in term 1.
I20260925 09:28:46.027673 22259 leader_election.cc:302] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 32bdcc62fe7c475ab6c79751686aea71, de45e69449884b6e98fe25b6e3dc61e6, f4a37fc526654e7aa1548c3971dacb35; no voters:
I20260925 09:28:46.027973 22538 raft_consensus.cc:2781] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:28:46.028368 22538 raft_consensus.cc:686] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [term 1 LEADER]: Becoming Leader. State: Replica: f4a37fc526654e7aa1548c3971dacb35, State: Running, Role: LEADER
I20260925 09:28:46.028806 22538 consensus_queue.cc:237] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de45e69449884b6e98fe25b6e3dc61e6" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 42295 } } peers { permanent_uuid: "f4a37fc526654e7aa1548c3971dacb35" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42971 } } peers { permanent_uuid: "32bdcc62fe7c475ab6c79751686aea71" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44427 } }
W20260925 09:28:46.031613 22235 tablet.cc:2369] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:28:46.033390 22020 catalog_manager.cc:5495] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4a37fc526654e7aa1548c3971dacb35 (127.10.114.66). New cstate: current_term: 1 leader_uuid: "f4a37fc526654e7aa1548c3971dacb35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de45e69449884b6e98fe25b6e3dc61e6" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 42295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4a37fc526654e7aa1548c3971dacb35" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32bdcc62fe7c475ab6c79751686aea71" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44427 } health_report { overall_health: UNKNOWN } } }
W20260925 09:28:46.041002 22378 tablet.cc:2369] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:28:46.053092 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:46.047907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46614 (local address 127.10.114.66:42971)
0925 09:28:46.048083 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:46.048089 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:46.048140 (+ 51us) server_negotiation.cc:407] Connection header received
0925 09:28:46.048356 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:46.048360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:46.048424 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:46.048561 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:28:46.048922 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:46.049601 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:46.050989 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:46.051364 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:46.051489 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:46.052062 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:28:46.052592 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:28:46.052723 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:46.052808 (+ 85us) server_negotiation.cc:299] Negotiation successful
0925 09:28:46.052881 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 09:28:46.061455 22466 raft_consensus.cc:1261] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6 [term 1 FOLLOWER]: Refusing update from remote peer f4a37fc526654e7aa1548c3971dacb35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:28:46.061457 22189 raft_consensus.cc:1261] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71 [term 1 FOLLOWER]: Refusing update from remote peer f4a37fc526654e7aa1548c3971dacb35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:28:46.062573 22538 consensus_queue.cc:1035] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de45e69449884b6e98fe25b6e3dc61e6" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 42295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:28:46.062919 22541 consensus_queue.cc:1035] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32bdcc62fe7c475ab6c79751686aea71" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 44427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:28:46.072721 22512 tablet.cc:2369] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:28:46.085110 22020 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:28:46.085421 22020 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:28:46.089180 22020 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7fbc166847564cb48c06286bfba578f3: Sending DeleteTablet for 3 replicas of tablet f00ad05feefb4922b848d18ac06e73ce
I20260925 09:28:46.090168 22446 tablet_service.cc:1514] Processing DeleteTablet for tablet f00ad05feefb4922b848d18ac06e73ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:28:46 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:43792
I20260925 09:28:46.090919 22554 tablet_replica.cc:330] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6: stopping tablet replica
I20260925 09:28:46.091303 22312 tablet_service.cc:1514] Processing DeleteTablet for tablet f00ad05feefb4922b848d18ac06e73ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:28:46 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:46602
I20260925 09:28:46.091325 22169 tablet_service.cc:1514] Processing DeleteTablet for tablet f00ad05feefb4922b848d18ac06e73ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:28:46 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:59420
I20260925 09:28:46.091660 22554 raft_consensus.cc:2229] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:28:46.091748 22168 tablet_service.cc:1514] Processing DeleteTablet for tablet f00ad05feefb4922b848d18ac06e73ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:28:46 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:59420
I20260925 09:28:46.092250 22554 raft_consensus.cc:2258] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:28:46.093783 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 22101
I20260925 09:28:46.094880 22554 ts_tablet_manager.cc:1905] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:28:46.103983 22554 ts_tablet_manager.cc:1918] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:28:46.104159 22556 tablet_replica.cc:330] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71: stopping tablet replica
I20260925 09:28:46.104190 22554 log.cc:1196] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/f00ad05feefb4922b848d18ac06e73ce
I20260925 09:28:46.104689 22554 ts_tablet_manager.cc:1939] T f00ad05feefb4922b848d18ac06e73ce P de45e69449884b6e98fe25b6e3dc61e6: Deleting consensus metadata
I20260925 09:28:46.104728 22556 raft_consensus.cc:2229] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:28:46.105247 22556 raft_consensus.cc:2258] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:28:46.106529 22005 catalog_manager.cc:4841] TS de45e69449884b6e98fe25b6e3dc61e6 (127.10.114.67:42295): tablet f00ad05feefb4922b848d18ac06e73ce (table test-table [id=1031ec61258e411f9cf41ae8ae51a781]) successfully deleted
I20260925 09:28:46.107122 22556 ts_tablet_manager.cc:1905] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:28:46.108909 22555 tablet_replica.cc:330] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35: stopping tablet replica
I20260925 09:28:46.109362 22555 raft_consensus.cc:2229] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:28:46.110040 22555 raft_consensus.cc:2258] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:28:46.113272 22555 ts_tablet_manager.cc:1905] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:28:46.113569 22556 ts_tablet_manager.cc:1918] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:28:46.113765 22556 log.cc:1196] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/f00ad05feefb4922b848d18ac06e73ce
I20260925 09:28:46.114333 22556 ts_tablet_manager.cc:1939] T f00ad05feefb4922b848d18ac06e73ce P 32bdcc62fe7c475ab6c79751686aea71: Deleting consensus metadata
I20260925 09:28:46.116216 22006 catalog_manager.cc:4841] TS 32bdcc62fe7c475ab6c79751686aea71 (127.10.114.65:44427): tablet f00ad05feefb4922b848d18ac06e73ce (table test-table [id=1031ec61258e411f9cf41ae8ae51a781]) successfully deleted
W20260925 09:28:46.117074 22006 catalog_manager.cc:4808] TS 32bdcc62fe7c475ab6c79751686aea71 (127.10.114.65:44427): delete failed for tablet f00ad05feefb4922b848d18ac06e73ce because the tablet was not found. No further retry: Not found: Tablet not found: f00ad05feefb4922b848d18ac06e73ce
I20260925 09:28:46.120594 22555 ts_tablet_manager.cc:1918] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:28:46.120754 22555 log.cc:1196] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/f00ad05feefb4922b848d18ac06e73ce
I20260925 09:28:46.121284 22555 ts_tablet_manager.cc:1939] T f00ad05feefb4922b848d18ac06e73ce P f4a37fc526654e7aa1548c3971dacb35: Deleting consensus metadata
Sep 25 09:28:46 dist-test-slave-9pj8 krb5kdc[21545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:28:46.122926 22006 catalog_manager.cc:4841] TS f4a37fc526654e7aa1548c3971dacb35 (127.10.114.66:42971): tablet f00ad05feefb4922b848d18ac06e73ce (table test-table [id=1031ec61258e411f9cf41ae8ae51a781]) successfully deleted
I20260925 09:28:46.128942 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:46.095253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59436 (local address 127.10.114.65:44427)
0925 09:28:46.095410 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:46.095418 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:46.108275 (+ 12857us) server_negotiation.cc:407] Connection header received
0925 09:28:46.108466 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:46.108473 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:46.108572 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:46.108685 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:28:46.110542 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:46.111757 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:46.114148 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:46.114867 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:46.114974 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:46.122833 (+ 7859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:28:46.122846 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:28:46.122863 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:28:46.122912 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:28:46.126161 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:46.126895 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:46.126902 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:46.126905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:46.127033 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:46.127390 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:46.127397 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:46.127400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:46.127736 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:28:46.128075 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:46.128365 (+ 290us) server_negotiation.cc:299] Negotiation successful
0925 09:28:46.128576 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 09:28:46.129525 22229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:28:46.220265 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 22101
I20260925 09:28:46.244539 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 22240
Sep 25 09:28:46 dist-test-slave-9pj8 krb5kdc[21545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.66@KRBTEST.COM
I20260925 09:28:46.261667 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:46.245429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46630 (local address 127.10.114.66:42971)
0925 09:28:46.245621 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:46.245629 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:46.245650 (+ 21us) server_negotiation.cc:407] Connection header received
0925 09:28:46.245718 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:46.245722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:46.245801 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:46.245903 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:28:46.247522 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:46.248468 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:46.249938 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:46.250351 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:46.250518 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:46.255045 (+ 4527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:28:46.255063 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:28:46.255079 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:28:46.255126 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:28:46.258411 (+ 3285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:46.259317 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:46.259339 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:46.259342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:46.259500 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:46.259972 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:46.259979 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:46.259984 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:46.260375 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:28:46.260752 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:46.261059 (+ 307us) server_negotiation.cc:299] Negotiation successful
0925 09:28:46.261281 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:28:46.262439 22372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:28:46.376659 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 22240
I20260925 09:28:46.395099 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 22383
Sep 25 09:28:46 dist-test-slave-9pj8 krb5kdc[21545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.67@KRBTEST.COM
I20260925 09:28:46.409972 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:28:46.395872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43794 (local address 127.10.114.67:42295)
0925 09:28:46.396064 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 09:28:46.396071 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:28:46.396141 (+ 70us) server_negotiation.cc:407] Connection header received
0925 09:28:46.396355 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:28:46.396366 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:28:46.396446 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:28:46.396552 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:28:46.397784 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:46.398511 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:46.399854 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:28:46.400261 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:28:46.400395 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:28:46.404552 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:28:46.404565 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:28:46.404580 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:28:46.404611 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:28:46.407186 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:46.407914 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:46.407921 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:46.407924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:46.408048 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:28:46.408437 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:28:46.408442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:28:46.408444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:28:46.408787 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:28:46.409071 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0925 09:28:46.409404 (+ 333us) server_negotiation.cc:299] Negotiation successful
0925 09:28:46.409636 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 09:28:46.410562 22506 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:28:46.497962 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 22383
I20260925 09:28:46.517776 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 21989
I20260925 09:28:46.518967 22051 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:28:46.626258 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 21989
I20260925 09:28:46.649859 10697 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 09:28:46.985648 10697 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 09:28:46.985855 10697 mini_ranger.cc:66] Stopping Ranger...
I20260925 09:28:47.331435 10697 mini_ranger.cc:68] Stopped Ranger
2026-09-25 09:28:47.331 UTC [21575] LOG: received smart shutdown request
2026-09-25 09:28:47.335 UTC [21575] LOG: background worker "logical replication launcher" (PID 21584) exited with exit code 1
2026-09-25 09:28:47.335 UTC [21579] LOG: shutting down
2026-09-25 09:28:47.500 UTC [21575] LOG: database system is shut down
2026-09-25T09:28:47Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46852 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:28:47 dist-test-slave-9pj8 krb5kdc[22570](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:28:47 dist-test-slave-9pj8 krb5kdc[22570](info): set up 2 sockets
Sep 25 09:28:47 dist-test-slave-9pj8 krb5kdc[22570](info): commencing operation
krb5kdc: starting...
W20260925 09:28:48.692394 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.005s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:28:48 dist-test-slave-9pj8 krb5kdc[22570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:28:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:28:48Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.114.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.114.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.114.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/rangeradmin_127.10.114.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.114.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/rangeradmin_127.10.114.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.114.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.114.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.114.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/rangerlookup_127.10.114.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.114.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/rangerlookup_127.10.114.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.114.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.114.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.114.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/HTTP_127.10.114.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.114.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/HTTP_127.10.114.84@KRBTEST.COM.keytab.
I20260925 09:28:48.999178 10697 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskMvTxiP/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/postgres -l logfile start
2026-09-25 09:28:51.397 UTC [22600] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 09:28:51.397 UTC [22600] LOG: listening on IPv4 address "127.10.114.84", port 53579
2026-09-25 09:28:51.401 UTC [22600] LOG: listening on Unix socket "/tmp/.s.PGSQL.53579"
2026-09-25 09:28:51.422 UTC [22603] LOG: database system was shut down at 2026-09-25 09:28:50 UTC
2026-09-25 09:28:51.425 UTC [22600] LOG: database system is ready to accept connections
I20260925 09:28:52.360122 10697 mini_postgres.cc:96] Postgres bound to 53579
2026-09-25 09:28:52.381 UTC [22611] FATAL: database "slave" does not exist
127.10.114.84:53579 - accepting connections
I20260925 09:28:52.381901 10697 mini_ranger.cc:161] Starting Ranger...
I20260925 09:28:52.405434 10697 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 09:28:52.851226 10697 mini_ranger.cc:87] Created ranger Postgres database
I20260925 09:28:52.851366 10697 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ranger-admin
2026-09-25 09:28:53,142 [I] DB FLAVOR :POSTGRES
2026-09-25 09:28:53,143 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 09:28:53,143 [I] Checking connection..
2026-09-25 09:28:53,143 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 09:28:53,399 [I] Checking connection passed.
2026-09-25 09:28:53,399 [I] --------- Verifying version history table ---------
2026-09-25 09:28:53,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 09:28:53,646 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 09:28:53,646 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 09:28:53,897 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 09:28:53,897 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 09:28:53,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 09:28:54.128 UTC [22675] WARNING: there is no transaction in progress
2026-09-25 09:28:54,134 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 09:28:54,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 09:28:54,366 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 09:28:54,366 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 09:28:54,366 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 09:28:54,598 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 09:28:54,838 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9pj8.c.gcp-upstream.internal','N') ;"
2026-09-25 09:28:55,068 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 09:28:55,318 [I] Table x_portal_user does not exist in database ranger
2026-09-25 09:28:55,319 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 09:28:55,565 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 09:28:55,565 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 09:28:55,812 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 09:28:56,061 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 09:28:56,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 09:28:56.971 UTC [22783] WARNING: there is no transaction in progress
2026-09-25 09:28:56.987 UTC [22783] WARNING: there is no transaction in progress
2026-09-25 09:28:57.002 UTC [22783] WARNING: there is no transaction in progress
2026-09-25 09:28:57.018 UTC [22783] WARNING: there is no transaction in progress
2026-09-25 09:28:57.034 UTC [22783] WARNING: there is no transaction in progress
2026-09-25 09:28:57.053 UTC [22783] WARNING: there is no transaction in progress
2026-09-25 09:28:57.113 UTC [22783] WARNING: there is no transaction in progress
2026-09-25 09:28:57.119 UTC [22783] WARNING: there is no transaction in progress
2026-09-25 09:28:57.133 UTC [22783] WARNING: there is no transaction in progress
2026-09-25 09:28:57.142 UTC [22783] WARNING: there is no transaction in progress
2026-09-25 09:28:57.152 UTC [22783] WARNING: there is no transaction in progress
2026-09-25 09:28:57.166 UTC [22783] WARNING: there is no transaction in progress
2026-09-25 09:28:57.175 UTC [22783] WARNING: there is no transaction in progress
2026-09-25 09:28:57.182 UTC [22783] WARNING: there is no transaction in progress
2026-09-25 09:28:57.188 UTC [22783] WARNING: there is no transaction in progress
2026-09-25 09:28:57.195 UTC [22783] WARNING: there is no transaction in progress
2026-09-25 09:28:57,793 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 09:28:57,794 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 09:28:58,003 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 09:28:58,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 09:28:58,240 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 09:28:58,240 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 09:28:58,481 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 09:28:58,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 09:28:58,726 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 09:28:58,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9pj8.c.gcp-upstream.internal';"
2026-09-25 09:28:59,178 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 09:28:59,178 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 09:28:59,429 [I] DB_PATCHES have already been applied
I20260925 09:28:59.435303 10697 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ranger-admin:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskMvTxiP/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 09:28:59.435406 10697 mini_ranger.cc:193] Using host: 127.10.114.84
I20260925 09:28:59.449653 10697 mini_ranger.cc:227] Ranger admin URL: http://127.10.114.84:54645
Sep 25, 2026 9:28:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 9:28:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 9:28:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 9:28:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 9:28:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 9:29:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 9:29:00 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 09:29:00.532860 10697 mini_ranger.cc:160] Time spent starting Ranger: real 8.151s user 0.008s sys 0.028s
Sep 25, 2026 9:29:20 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 9:29:20 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 25, 2026 9:29:20 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 9:29:20 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 9:29:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 9:29:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 9:29:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 9:29:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 9:29:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 9:29:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 9:29:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 9:29:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 9:29:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 9:29:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 9:29:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 9:29:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 9:29:20 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 9:29:21 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 09:29:22.022027 10697 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.114.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.114.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.114.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/rangerkms_127.10.114.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.114.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/rangerkms_127.10.114.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.114.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.114.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.114.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/HTTP_127.10.114.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.114.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/HTTP_127.10.114.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 09:29:22 dist-test-slave-9pj8 krb5kdc[22570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328562, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 09:29:22.174192 10697 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 09:29:22.192950 10697 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 09:29:23.033150 10697 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 09:29:23.033246 10697 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ranger-kms
I20260925 09:29:23.033288 10697 mini_ranger_kms.cc:227] Using postgres at 127.10.114.84:53579
2026-09-25 09:29:23,372 [I] DB FLAVOR :POSTGRES
2026-09-25 09:29:23,373 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 09:29:23,373 [I] Checking connection
2026-09-25 09:29:23,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 09:29:23,622 [I] connection success
2026-09-25 09:29:23,623 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 09:29:23,623 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 09:29:23,623 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 09:29:23,850 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 09:29:23,851 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 09:29:23,851 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 09:29:23,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.114.84:53579/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 09:29:24,097 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 09:29:24.579319 10697 mini_ranger_kms.cc:310] Created kms service
I20260925 09:29:24.758728 10697 mini_ranger_kms.cc:326] Created kudu user
I20260925 09:29:24.806697 10697 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 09:29:24 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328562, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
I20260925 09:29:25.194139 10697 mini_ranger_kms.cc:384] Added ranger policy
I20260925 09:29:25.194466 10697 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ranger-kms:/tmp/dist-test-taskMvTxiP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskMvTxiP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 09:29:25.194532 10697 mini_ranger_kms.cc:240] Using host: 127.10.114.84
I20260925 09:29:25.209631 10697 mini_ranger_kms.cc:276] Ranger KMS PID: 22963
I20260925 09:29:25.209755 10697 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.114.84:56133
Sep 25, 2026 9:29:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 9:29:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 9:29:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 9:29:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 9:29:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 9:29:26 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260925 09:29:27.472482 10697 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.298s user 0.044s sys 0.300s
I20260925 09:29:27.472736 10697 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 09:29:27.472921 10697 mini_ranger_kms.cc:401] 127.10.114.84:56133/kms/v1/keys
Sep 25 09:29:27 dist-test-slave-9pj8 krb5kdc[22570](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790328567, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.114.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 25 09:29:27 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328567, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.114.84@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
Sep 25 09:29:28 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328567, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.114.84@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
Sep 25 09:29:30 dist-test-slave-9pj8 krb5kdc[22570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:29:30.582914 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:39417
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:39417
--ranger_config_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.114.84:56133/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:29:30.919507 23014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:30.919919 23014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:30.920080 23014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:30.927389 23014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:29:30.927486 23014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:30.927554 23014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:30.927655 23014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:29:30.927717 23014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:29:30.937901 23014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.114.84:56133/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:39417
--ranger_config_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:39417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:30.939234 23014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:30.940637 23014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:30.954435 23020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:30.954490 23022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:30.954901 23014 server_base.cc:1034] running on GCE node
W20260925 09:29:30.955255 23019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:30.956143 23014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:30.957553 23014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:30.958781 23014 hybrid_clock.cc:648] HybridClock initialized: now 1790328570958728 us; error 49 us; skew 500 ppm
I20260925 09:29:30.959141 23014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:30 dist-test-slave-9pj8 krb5kdc[22570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328570, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:30.970188 23014 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:29:30.971956 23014 webserver.cc:460] Webserver started at http://127.10.114.126:41139/ using document root <none> and password file <none>
I20260925 09:29:30.972646 23014 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:30.972767 23014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:29:30.973111 23014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 09:29:30 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328570, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
I20260925 09:29:31.036307 23014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "31d38d18783c4b289f8ff541b564aa0b"
format_stamp: "Formatted at 2026-09-25 09:29:31 on dist-test-slave-9pj8"
server_key: "49ff2a24a38de8297c7095000c8e7b17"
server_key_iv: "0fa7eec50c777142aeac6b4c7227e33c"
server_key_version: "kuduclusterkey@0"
I20260925 09:29:31.037194 23014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "31d38d18783c4b289f8ff541b564aa0b"
format_stamp: "Formatted at 2026-09-25 09:29:31 on dist-test-slave-9pj8"
server_key: "49ff2a24a38de8297c7095000c8e7b17"
server_key_iv: "0fa7eec50c777142aeac6b4c7227e33c"
server_key_version: "kuduclusterkey@0"
I20260925 09:29:31.042688 23014 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 09:29:31.061626 23029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:31.063482 23014 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260925 09:29:31.063686 23014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "31d38d18783c4b289f8ff541b564aa0b"
format_stamp: "Formatted at 2026-09-25 09:29:31 on dist-test-slave-9pj8"
server_key: "49ff2a24a38de8297c7095000c8e7b17"
server_key_iv: "0fa7eec50c777142aeac6b4c7227e33c"
server_key_version: "kuduclusterkey@0"
I20260925 09:29:31.063995 23014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:31.102043 23014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:31.107024 23014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:31.107406 23014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:31.127951 23014 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:39417
I20260925 09:29:31.127977 23080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:39417 every 8 connection(s)
I20260925 09:29:31.129767 23014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:29:31.133981 23081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:29:31.138196 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 23014
I20260925 09:29:31.138445 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:29:31.139348 10697 external_mini_cluster.cc:1428] Setting key 63d5000e89a7c203565abf2a26a4513d
I20260925 09:29:31.150480 23081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b: Bootstrap starting.
I20260925 09:29:31.155545 23081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b: Neither blocks nor log segments found. Creating new log.
I20260925 09:29:31.157413 23081 log.cc:824] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b: Log is configured to *not* fsync() on all Append() calls
I20260925 09:29:31.160430 23081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b: No bootstrap required, opened a new log
I20260925 09:29:31.169608 23081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d38d18783c4b289f8ff541b564aa0b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39417 } }
I20260925 09:29:31.169988 23081 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:29:31.170097 23081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31d38d18783c4b289f8ff541b564aa0b, State: Initialized, Role: FOLLOWER
I20260925 09:29:31.170754 23081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d38d18783c4b289f8ff541b564aa0b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39417 } }
I20260925 09:29:31.170969 23081 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:29:31.171096 23081 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:29:31.171317 23081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:29:31.174474 23081 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d38d18783c4b289f8ff541b564aa0b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39417 } }
Sep 25 09:29:31 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:29:31.175022 23081 leader_election.cc:302] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31d38d18783c4b289f8ff541b564aa0b; no voters:
I20260925 09:29:31.175678 23081 leader_election.cc:288] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:29:31.175817 23086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:29:31.179867 23086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [term 1 LEADER]: Becoming Leader. State: Replica: 31d38d18783c4b289f8ff541b564aa0b, State: Running, Role: LEADER
I20260925 09:29:31.180611 23086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d38d18783c4b289f8ff541b564aa0b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39417 } }
I20260925 09:29:31.185384 23081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:29:31.186164 23084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:31.141628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55456 (local address 127.10.114.126:39417)
0925 09:29:31.156962 (+ 15334us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:31.156979 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:31.157033 (+ 54us) server_negotiation.cc:407] Connection header received
0925 09:29:31.159685 (+ 2652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:31.159716 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:31.160260 (+ 544us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:31.160729 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:31.162380 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:31.164633 (+ 2253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:31.166578 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:31.167689 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:31.167916 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:31.176066 (+ 8150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:31.176079 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:31.176104 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:31.176163 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:31.180341 (+ 4178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:31.181249 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:31.181259 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:31.181269 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:31.181448 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:31.181895 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:31.181901 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:31.181904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:31.182640 (+ 736us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:31.184056 (+ 1416us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:31.184367 (+ 311us) server_negotiation.cc:299] Negotiation successful
0925 09:29:31.184770 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15082,"thread_start_us":14762,"threads_started":1}
I20260925 09:29:31.190529 23088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31d38d18783c4b289f8ff541b564aa0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d38d18783c4b289f8ff541b564aa0b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39417 } } }
I20260925 09:29:31.191094 23088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [sys.catalog]: This master's current role is: LEADER
I20260925 09:29:31.197245 23087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31d38d18783c4b289f8ff541b564aa0b. Latest consensus state: current_term: 1 leader_uuid: "31d38d18783c4b289f8ff541b564aa0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d38d18783c4b289f8ff541b564aa0b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39417 } } }
I20260925 09:29:31.197607 23087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [sys.catalog]: This master's current role is: LEADER
I20260925 09:29:31.198206 23092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:29:31.210825 23092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:29:31.211601 23081 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 09:29:31.232388 23092 catalog_manager.cc:1347] Generated new cluster ID: 2b5e685cf44946a9b6291172e4d921e1
I20260925 09:29:31.232537 23092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:29:31.252079 23092 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:29:31.254125 23092 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:29:31.263562 23092 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b: Generated new TSK 0
I20260925 09:29:31.264763 23092 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
I20260925 09:29:32.055291 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.114.84:56133/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39417
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:29:32 dist-test-slave-9pj8 krb5kdc[22570](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790328572, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260925 09:29:32.474398 23126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:32.474793 23126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:32.474993 23126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:32.486696 23126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:32.486867 23126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:32.487058 23126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:29:32.501825 23126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.114.84:56133/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:32.503489 23126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:32.505117 23126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:32.532495 23137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:32.534710 23136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:32.535434 23139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:32.535800 23126 server_base.cc:1034] running on GCE node
I20260925 09:29:32.536600 23126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:32.537647 23126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:32.538868 23126 hybrid_clock.cc:648] HybridClock initialized: now 1790328572538754 us; error 109 us; skew 500 ppm
I20260925 09:29:32.539206 23126 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:32 dist-test-slave-9pj8 krb5kdc[22570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328572, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:32.543017 23126 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:29:32.545100 23126 webserver.cc:460] Webserver started at http://127.10.114.65:39289/ using document root <none> and password file <none>
I20260925 09:29:32.545912 23126 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:32.546039 23126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:29:32.546350 23126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 09:29:32 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328572, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
I20260925 09:29:32.567494 23126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "1a668d7b33314865ae897e0f8c16b011"
format_stamp: "Formatted at 2026-09-25 09:29:32 on dist-test-slave-9pj8"
server_key: "206110127cf0c5b04e9724fae967c986"
server_key_iv: "d92f06d8f72a0e0cefdaed5752de1288"
server_key_version: "kuduclusterkey@0"
I20260925 09:29:32.568706 23126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "1a668d7b33314865ae897e0f8c16b011"
format_stamp: "Formatted at 2026-09-25 09:29:32 on dist-test-slave-9pj8"
server_key: "206110127cf0c5b04e9724fae967c986"
server_key_iv: "d92f06d8f72a0e0cefdaed5752de1288"
server_key_version: "kuduclusterkey@0"
I20260925 09:29:32.576463 23126 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260925 09:29:32.605876 23146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:32.607934 23126 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.004s sys 0.006s
I20260925 09:29:32.608184 23126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "1a668d7b33314865ae897e0f8c16b011"
format_stamp: "Formatted at 2026-09-25 09:29:32 on dist-test-slave-9pj8"
server_key: "206110127cf0c5b04e9724fae967c986"
server_key_iv: "d92f06d8f72a0e0cefdaed5752de1288"
server_key_version: "kuduclusterkey@0"
I20260925 09:29:32.608568 23126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:32.631039 23126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:32.637598 23126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:32.637984 23126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:32.639293 23126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:29:32.641105 23126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:29:32.641235 23126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260925 09:29:32.641381 23126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:29:32.641474 23126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:32.733671 23126 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:46785
I20260925 09:29:32.733692 23258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:46785 every 8 connection(s)
Sep 25 09:29:32 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328572, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
I20260925 09:29:32.735328 23126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:29:32.739741 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 23126
I20260925 09:29:32.739974 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:29:32.740703 10697 external_mini_cluster.cc:1428] Setting key 0a4b3a3856daef9a64bd0ed0c34de3ac
Sep 25 09:29:32 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328572, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.114.66@KRBTEST.COM; defaulting to no policy
I20260925 09:29:32.821453 23262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:32.741341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:47295 (local address 127.10.114.126:39417)
0925 09:29:32.782436 (+ 41095us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:32.782445 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:32.793079 (+ 10634us) server_negotiation.cc:407] Connection header received
0925 09:29:32.795407 (+ 2328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:32.795413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:32.795495 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:32.795567 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:32.800510 (+ 4943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:32.801642 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:32.804003 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:32.804600 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:32.804696 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:32.809343 (+ 4647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:32.809347 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:32.809351 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:32.809382 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:32.812985 (+ 3603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:32.818443 (+ 5458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:32.818449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:32.818452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:32.818641 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:32.819216 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:32.819221 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:32.819225 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:32.819594 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:32.819728 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:32.820823 (+ 1095us) server_negotiation.cc:299] Negotiation successful
0925 09:29:32.821059 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40939,"thread_start_us":41646,"threads_started":1}
I20260925 09:29:32.823962 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:32.738897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39417 (local address 127.10.114.65:47295)
0925 09:29:32.784013 (+ 45116us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:32.784075 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:32.795066 (+ 10991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:32.796169 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:32.796181 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:32.796833 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:32.797681 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:32.797704 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:32.801865 (+ 4161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:32.801871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:32.803824 (+ 1953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:32.803850 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:32.805036 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:32.805042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:32.805330 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:32.806112 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:32.806145 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:32.809095 (+ 2950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:32.816529 (+ 7434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:32.816551 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:32.816569 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:32.817147 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:32.818812 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:32.818817 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:32.818820 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:32.819081 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:32.819847 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:32.819855 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:32.820285 (+ 430us) client_negotiation.cc:769] Sending connection context
0925 09:29:32.822005 (+ 1720us) client_negotiation.cc:240] Negotiation successful
0925 09:29:32.822548 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44712,"thread_start_us":48810,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.66@KRBTEST.COM" created.
I20260925 09:29:32.826098 23259 heartbeater.cc:347] Connected to a master server at 127.10.114.126:39417
I20260925 09:29:32.826416 23259 heartbeater.cc:464] Registering TS with master...
I20260925 09:29:32.827292 23259 heartbeater.cc:511] Master 127.10.114.126:39417 requested a full tablet report, sending...
I20260925 09:29:32.829787 23046 ts_manager.cc:194] Registered new tserver with Master: 1a668d7b33314865ae897e0f8c16b011 (127.10.114.65:46785)
I20260925 09:29:32.831736 23046 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:47295
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
I20260925 09:29:32.866582 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.114.84:56133/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39417
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:29:33 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328572, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
W20260925 09:29:33.353137 23265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:33.353477 23265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:33.353657 23265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:33.360987 23265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:33.361145 23265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:33.361338 23265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:29:33.371380 23265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.114.84:56133/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:33.372707 23265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:33.374006 23265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:33.390098 23279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:33.391490 23280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:33.396535 23265 server_base.cc:1034] running on GCE node
W20260925 09:29:33.396874 23282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:33.397688 23265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:33.399020 23265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:33.400300 23265 hybrid_clock.cc:648] HybridClock initialized: now 1790328573400248 us; error 44 us; skew 500 ppm
I20260925 09:29:33.400714 23265 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:33 dist-test-slave-9pj8 krb5kdc[22570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328573, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:33.404946 23265 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:29:33.406911 23265 webserver.cc:460] Webserver started at http://127.10.114.66:37391/ using document root <none> and password file <none>
I20260925 09:29:33.407905 23265 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:33.408051 23265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:29:33.408424 23265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 09:29:33 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328573, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
I20260925 09:29:33.425660 23265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "e515c0373e714c40965bc88c90fa41a3"
format_stamp: "Formatted at 2026-09-25 09:29:33 on dist-test-slave-9pj8"
server_key: "7c0b04a63f9e0aef93515d8e195a750e"
server_key_iv: "6120e671f27153a7210cb97a5d0a3d7e"
server_key_version: "kuduclusterkey@0"
I20260925 09:29:33.426512 23265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "e515c0373e714c40965bc88c90fa41a3"
format_stamp: "Formatted at 2026-09-25 09:29:33 on dist-test-slave-9pj8"
server_key: "7c0b04a63f9e0aef93515d8e195a750e"
server_key_iv: "6120e671f27153a7210cb97a5d0a3d7e"
server_key_version: "kuduclusterkey@0"
I20260925 09:29:33.431995 23265 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260925 09:29:33.455977 23289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:33.457803 23265 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.004s sys 0.004s
I20260925 09:29:33.457985 23265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "e515c0373e714c40965bc88c90fa41a3"
format_stamp: "Formatted at 2026-09-25 09:29:33 on dist-test-slave-9pj8"
server_key: "7c0b04a63f9e0aef93515d8e195a750e"
server_key_iv: "6120e671f27153a7210cb97a5d0a3d7e"
server_key_version: "kuduclusterkey@0"
I20260925 09:29:33.458204 23265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:33.483047 23265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:33.487138 23265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:33.487421 23265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:33.488487 23265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:29:33.489977 23265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:29:33.490043 23265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:33.490202 23265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:29:33.490249 23265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:33.535135 23265 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:46743
I20260925 09:29:33.535156 23401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:46743 every 8 connection(s)
I20260925 09:29:33.536504 23265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:29:33.545374 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 23265
I20260925 09:29:33.545557 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:29:33.546178 10697 external_mini_cluster.cc:1428] Setting key 56212e8c15b420c5b97b77a433705f24
Sep 25 09:29:33 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328573, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:29:33.557417 23404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:33.539248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:55739 (local address 127.10.114.126:39417)
0925 09:29:33.539825 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:33.539834 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:33.540919 (+ 1085us) server_negotiation.cc:407] Connection header received
0925 09:29:33.542713 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:33.542719 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:33.542837 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:33.542957 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:33.544401 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:33.545154 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:33.546968 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:33.547661 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:33.547835 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:33.551371 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:33.551375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:33.551378 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:33.551407 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:33.554265 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:33.555046 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:33.555050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:33.555052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:33.555181 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:33.555640 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:33.555649 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:33.555652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:33.556003 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:33.556169 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:33.556835 (+ 666us) server_negotiation.cc:299] Negotiation successful
0925 09:29:33.557063 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":275,"threads_started":1}
I20260925 09:29:33.558506 23405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:33.539901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39417 (local address 127.10.114.66:55739)
0925 09:29:33.540766 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:33.540803 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:33.542426 (+ 1623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:33.543189 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:33.543197 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:33.543616 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:33.544238 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:33.544254 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:33.545323 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:33.545327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:33.546802 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:33.546817 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:33.547927 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:33.547934 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:33.548235 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:33.548903 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:33.548925 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:33.551152 (+ 2227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:33.554431 (+ 3279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:33.554446 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:33.554462 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:33.554895 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:33.555291 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:33.555297 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:33.555300 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:33.555516 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:33.556143 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:33.556151 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:33.556477 (+ 326us) client_negotiation.cc:769] Sending connection context
0925 09:29:33.556732 (+ 255us) client_negotiation.cc:240] Negotiation successful
0925 09:29:33.557220 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":342,"threads_started":1}
I20260925 09:29:33.560079 23402 heartbeater.cc:347] Connected to a master server at 127.10.114.126:39417
I20260925 09:29:33.560387 23402 heartbeater.cc:464] Registering TS with master...
I20260925 09:29:33.561017 23402 heartbeater.cc:511] Master 127.10.114.126:39417 requested a full tablet report, sending...
I20260925 09:29:33.562436 23046 ts_manager.cc:194] Registered new tserver with Master: e515c0373e714c40965bc88c90fa41a3 (127.10.114.66:46743)
I20260925 09:29:33.563337 23046 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:55739
WARNING: no policy specified for kudu/127.10.114.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
I20260925 09:29:33.600654 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.114.84:56133/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39417
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:29:33.835361 23259 heartbeater.cc:503] Master 127.10.114.126:39417 was elected leader, sending a full tablet report...
W20260925 09:29:33.915571 23408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:33.915876 23408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:33.916064 23408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:33.923030 23408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:33.923120 23408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:33.923260 23408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:29:33.932163 23408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.114.84:56133/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:33.933439 23408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:33.934674 23408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:33.947144 23414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:33.947157 23416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:33.947750 23408 server_base.cc:1034] running on GCE node
W20260925 09:29:33.947923 23413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:33.948565 23408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:33.949499 23408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:33.950685 23408 hybrid_clock.cc:648] HybridClock initialized: now 1790328573950651 us; error 31 us; skew 500 ppm
I20260925 09:29:33.950987 23408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:33 dist-test-slave-9pj8 krb5kdc[22570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328573, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:33.953964 23408 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:29:33.955245 23408 webserver.cc:460] Webserver started at http://127.10.114.67:44849/ using document root <none> and password file <none>
I20260925 09:29:33.955965 23408 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:33.956091 23408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:29:33.956382 23408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 09:29:33 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328573, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
I20260925 09:29:33.971355 23408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "f3061813deb64a3b8ad2435986327438"
format_stamp: "Formatted at 2026-09-25 09:29:33 on dist-test-slave-9pj8"
server_key: "2193ba8cb6e68c9ff840d56a738b4dd7"
server_key_iv: "7049845470c8b3ef6fdcb48b0564f259"
server_key_version: "kuduclusterkey@0"
I20260925 09:29:33.972169 23408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "f3061813deb64a3b8ad2435986327438"
format_stamp: "Formatted at 2026-09-25 09:29:33 on dist-test-slave-9pj8"
server_key: "2193ba8cb6e68c9ff840d56a738b4dd7"
server_key_iv: "7049845470c8b3ef6fdcb48b0564f259"
server_key_version: "kuduclusterkey@0"
I20260925 09:29:33.977429 23408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260925 09:29:33.993767 23423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:33.995217 23408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 09:29:33.995368 23408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "f3061813deb64a3b8ad2435986327438"
format_stamp: "Formatted at 2026-09-25 09:29:33 on dist-test-slave-9pj8"
server_key: "2193ba8cb6e68c9ff840d56a738b4dd7"
server_key_iv: "7049845470c8b3ef6fdcb48b0564f259"
server_key_version: "kuduclusterkey@0"
I20260925 09:29:33.995569 23408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:34.022258 23408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:34.025873 23408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:34.026178 23408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:34.027081 23408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:29:34.028414 23408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:29:34.028486 23408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:34.028637 23408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:29:34.028687 23408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:34.060122 23408 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:35235
I20260925 09:29:34.060174 23535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:35235 every 8 connection(s)
I20260925 09:29:34.061411 23408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:29:34.071152 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 23408
I20260925 09:29:34.071326 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:29:34.072032 10697 external_mini_cluster.cc:1428] Setting key 0bb990a69ccca6b5d26aff4059a167fd
Sep 25 09:29:34 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328573, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:29:34.082072 23538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:34.063997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:46991 (local address 127.10.114.126:39417)
0925 09:29:34.064489 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:34.064495 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:34.067627 (+ 3132us) server_negotiation.cc:407] Connection header received
0925 09:29:34.067684 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:34.067687 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:34.067747 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:34.067890 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:34.069271 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:34.070473 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:34.072759 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:34.073237 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:34.073397 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:34.076947 (+ 3550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:34.076952 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:34.076955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:34.076984 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:34.079625 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:34.080290 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:34.080294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:34.080296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:34.080420 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:34.080776 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:34.080779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:34.080781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:34.081022 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:34.081127 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:34.081659 (+ 532us) server_negotiation.cc:299] Negotiation successful
0925 09:29:34.081807 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":183,"threads_started":1}
I20260925 09:29:34.082942 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:34.064579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39417 (local address 127.10.114.67:46991)
0925 09:29:34.065309 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:34.065367 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:34.067339 (+ 1972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:34.068086 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:34.068094 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:34.068536 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:34.069118 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:34.069141 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:34.070596 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:34.070600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:34.072624 (+ 2024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:34.072635 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:34.073491 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:34.073499 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:34.073802 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:34.074605 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:34.074625 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:34.076768 (+ 2143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:34.079739 (+ 2971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:34.079745 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:34.079754 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:34.080184 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:34.080512 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:34.080515 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:34.080517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:34.080683 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:34.081119 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:34.081124 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:34.081377 (+ 253us) client_negotiation.cc:769] Sending connection context
0925 09:29:34.081573 (+ 196us) client_negotiation.cc:240] Negotiation successful
0925 09:29:34.081884 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":261,"threads_started":1}
I20260925 09:29:34.084174 23536 heartbeater.cc:347] Connected to a master server at 127.10.114.126:39417
I20260925 09:29:34.084435 23536 heartbeater.cc:464] Registering TS with master...
I20260925 09:29:34.085078 23536 heartbeater.cc:511] Master 127.10.114.126:39417 requested a full tablet report, sending...
I20260925 09:29:34.086786 23046 ts_manager.cc:194] Registered new tserver with Master: f3061813deb64a3b8ad2435986327438 (127.10.114.67:35235)
I20260925 09:29:34.087625 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:29:34.087972 23046 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:46991
I20260925 09:29:34.089511 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 23126
Sep 25 09:29:34 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:29:34.105151 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:34.090407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40748 (local address 127.10.114.65:46785)
0925 09:29:34.091003 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:34.091010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:34.091038 (+ 28us) server_negotiation.cc:407] Connection header received
0925 09:29:34.091106 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:34.091118 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:34.091313 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:34.091500 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:34.092983 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:34.093895 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:34.095360 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:34.095960 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:34.096091 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:34.100368 (+ 4277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:34.100378 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:34.100408 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:34.100452 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:34.102847 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:34.103451 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:34.103457 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:34.103459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:34.103585 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:34.103955 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:34.103958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:34.103960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:34.104262 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:34.104474 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:34.104680 (+ 206us) server_negotiation.cc:299] Negotiation successful
0925 09:29:34.104878 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":260,"threads_started":1}
I20260925 09:29:34.106231 23254 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:29:34.193157 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 23126
I20260925 09:29:34.209649 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 23265
Sep 25 09:29:34 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.66@KRBTEST.COM
I20260925 09:29:34.224826 23543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:34.210490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38560 (local address 127.10.114.66:46743)
0925 09:29:34.211017 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:34.211024 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:34.211049 (+ 25us) server_negotiation.cc:407] Connection header received
0925 09:29:34.211123 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:34.211130 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:34.211345 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:34.211475 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:34.212670 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:34.213580 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:34.214773 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:34.215285 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:34.215377 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:34.219461 (+ 4084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:34.219472 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:34.219487 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:34.219526 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:34.222370 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:34.222987 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:34.222994 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:34.222996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:34.223112 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:34.223428 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:34.223431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:34.223433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:34.223744 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:34.224033 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:34.224230 (+ 197us) server_negotiation.cc:299] Negotiation successful
0925 09:29:34.224415 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":274,"threads_started":1}
I20260925 09:29:34.226001 23397 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:29:34.307094 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 23265
I20260925 09:29:34.322448 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 23408
Sep 25 09:29:34 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.67@KRBTEST.COM
I20260925 09:29:34.338588 23545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:34.323414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51668 (local address 127.10.114.67:35235)
0925 09:29:34.324073 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:34.324080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:34.324106 (+ 26us) server_negotiation.cc:407] Connection header received
0925 09:29:34.324167 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:34.324172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:34.324475 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:34.324665 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:34.326463 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:34.327999 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:34.329217 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:34.329772 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:34.329897 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:34.334032 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:34.334043 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:34.334056 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:34.334089 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:34.336550 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:34.337084 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:34.337090 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:34.337092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:34.337201 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:34.337488 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:34.337491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:34.337493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:34.337769 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:34.337971 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:34.338176 (+ 205us) server_negotiation.cc:299] Negotiation successful
0925 09:29:34.338338 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":369,"threads_started":1}
I20260925 09:29:34.339901 23531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:29:34.419476 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 23408
I20260925 09:29:34.434680 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 23014
I20260925 09:29:34.435999 23076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:29:34.542658 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 23014
I20260925 09:29:34.563076 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:39417
--webserver_interface=127.10.114.126
--webserver_port=41139
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:39417
--ranger_config_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.114.84:56133/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.114.84:56133/kms with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:29:34.873211 23548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:34.873479 23548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:34.873632 23548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:34.880769 23548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:29:34.880909 23548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:34.880978 23548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:34.881055 23548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:29:34.881102 23548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:29:34.890842 23548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.114.84:56133/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:39417
--ranger_config_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:39417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=41139
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:34.892067 23548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:34.893357 23548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:34.904754 23553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:34.904789 23556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:34.904765 23554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:34.905933 23548 server_base.cc:1034] running on GCE node
I20260925 09:29:34.906672 23548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:34.908020 23548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:34.909245 23548 hybrid_clock.cc:648] HybridClock initialized: now 1790328574909188 us; error 65 us; skew 500 ppm
I20260925 09:29:34.909567 23548 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:34 dist-test-slave-9pj8 krb5kdc[22570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328574, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:34.912536 23548 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:29:34.913952 23548 webserver.cc:460] Webserver started at http://127.10.114.126:41139/ using document root <none> and password file <none>
I20260925 09:29:34.914546 23548 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:34.914672 23548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 09:29:34 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328574, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
I20260925 09:29:34.976263 23548 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:29:34.979390 23564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:34.980882 23548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:29:34.981071 23548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "31d38d18783c4b289f8ff541b564aa0b"
format_stamp: "Formatted at 2026-09-25 09:29:31 on dist-test-slave-9pj8"
server_key: "49ff2a24a38de8297c7095000c8e7b17"
server_key_iv: "0fa7eec50c777142aeac6b4c7227e33c"
server_key_version: "kuduclusterkey@0"
I20260925 09:29:34.982016 23548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:35.012912 23548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:35.017019 23548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:35.017324 23548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:35.035681 23548 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:39417
I20260925 09:29:35.035697 23615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:39417 every 8 connection(s)
I20260925 09:29:35.037166 23548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:29:35.043757 23616 sys_catalog.cc:263] Verifying existing consensus state
I20260925 09:29:35.044909 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 23548
I20260925 09:29:35.045823 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.114.84:56133/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:46785
--local_ip_for_outbound_sockets=127.10.114.65
--tserver_master_addrs=127.10.114.126:39417
--webserver_port=39289
--webserver_interface=127.10.114.65
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.114.84:56133/kms with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:29:35.046943 23616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b: Bootstrap starting.
I20260925 09:29:35.070429 23616 log.cc:824] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b: Log is configured to *not* fsync() on all Append() calls
I20260925 09:29:35.080816 23616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 09:29:35.081451 23616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b: Bootstrap complete.
I20260925 09:29:35.093945 23616 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d38d18783c4b289f8ff541b564aa0b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39417 } }
I20260925 09:29:35.095041 23616 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31d38d18783c4b289f8ff541b564aa0b, State: Initialized, Role: FOLLOWER
I20260925 09:29:35.095721 23616 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d38d18783c4b289f8ff541b564aa0b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39417 } }
I20260925 09:29:35.096004 23616 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:29:35.096110 23616 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:29:35.096277 23616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:29:35.101898 23616 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d38d18783c4b289f8ff541b564aa0b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39417 } }
I20260925 09:29:35.102491 23616 leader_election.cc:302] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31d38d18783c4b289f8ff541b564aa0b; no voters:
I20260925 09:29:35.103202 23616 leader_election.cc:288] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 09:29:35.103356 23620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:29:35.104497 23620 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [term 2 LEADER]: Becoming Leader. State: Replica: 31d38d18783c4b289f8ff541b564aa0b, State: Running, Role: LEADER
I20260925 09:29:35.105027 23620 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d38d18783c4b289f8ff541b564aa0b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39417 } }
I20260925 09:29:35.105254 23616 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:29:35.107571 23621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "31d38d18783c4b289f8ff541b564aa0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d38d18783c4b289f8ff541b564aa0b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39417 } } }
I20260925 09:29:35.107577 23622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31d38d18783c4b289f8ff541b564aa0b. Latest consensus state: current_term: 2 leader_uuid: "31d38d18783c4b289f8ff541b564aa0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d38d18783c4b289f8ff541b564aa0b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39417 } } }
I20260925 09:29:35.108029 23622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [sys.catalog]: This master's current role is: LEADER
I20260925 09:29:35.108026 23621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b [sys.catalog]: This master's current role is: LEADER
I20260925 09:29:35.109195 23626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:29:35.125587 23626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:29:35.127000 23626 catalog_manager.cc:1259] Loaded cluster ID: 2b5e685cf44946a9b6291172e4d921e1
I20260925 09:29:35.127151 23626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:29:35.131006 23626 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:29:35.132540 23626 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b: Loaded TSK: 0
I20260925 09:29:35.134035 23626 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:29:35.137338 23616 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260925 09:29:35.472626 23618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:35.473030 23618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:35.473246 23618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:35.484472 23618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:35.484650 23618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:35.484899 23618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:29:35.499946 23618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.114.84:56133/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:46785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=39289
--tserver_master_addrs=127.10.114.126:39417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:35.501711 23618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:35.503353 23618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:35.534134 23654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:35.536087 23653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:35.537829 23618 server_base.cc:1034] running on GCE node
W20260925 09:29:35.538653 23657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:35.539814 23618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:35.541047 23618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:35.542331 23618 hybrid_clock.cc:648] HybridClock initialized: now 1790328575542267 us; error 50 us; skew 500 ppm
I20260925 09:29:35.542835 23618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:35 dist-test-slave-9pj8 krb5kdc[22570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328575, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:35.546950 23618 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:29:35.564205 23618 webserver.cc:460] Webserver started at http://127.10.114.65:39289/ using document root <none> and password file <none>
I20260925 09:29:35.565223 23618 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:35.565404 23618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 09:29:35 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328575, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
I20260925 09:29:35.627153 23618 fs_manager.cc:709] Time spent opening directory manager: real 0.041s user 0.007s sys 0.032s
I20260925 09:29:35.636886 23664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:35.638429 23618 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.000s sys 0.009s
I20260925 09:29:35.638641 23618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "1a668d7b33314865ae897e0f8c16b011"
format_stamp: "Formatted at 2026-09-25 09:29:32 on dist-test-slave-9pj8"
server_key: "206110127cf0c5b04e9724fae967c986"
server_key_iv: "d92f06d8f72a0e0cefdaed5752de1288"
server_key_version: "kuduclusterkey@0"
I20260925 09:29:35.639747 23618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:35.683070 23618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:35.688138 23618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:35.688413 23618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:35.689486 23618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:29:35.691084 23618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:29:35.691205 23618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:35.691320 23618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:29:35.691387 23618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:35.783499 23618 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:46785
I20260925 09:29:35.783551 23777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:46785 every 8 connection(s)
I20260925 09:29:35.785092 23618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:29:35.789232 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 23618
I20260925 09:29:35.790020 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.114.84:56133/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:46743
--local_ip_for_outbound_sockets=127.10.114.66
--tserver_master_addrs=127.10.114.126:39417
--webserver_port=37391
--webserver_interface=127.10.114.66
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.114.84:56133/kms with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:29:35 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328575, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:29:35.834318 23781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:35.790435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:56765 (local address 127.10.114.126:39417)
0925 09:29:35.804431 (+ 13996us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:35.804453 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:35.805806 (+ 1353us) server_negotiation.cc:407] Connection header received
0925 09:29:35.810058 (+ 4252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:35.810080 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:35.810587 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:35.811017 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:35.814319 (+ 3302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:35.816563 (+ 2244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:35.819330 (+ 2767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:35.820520 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:35.820717 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:35.825396 (+ 4679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:35.825405 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:35.825423 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:35.825472 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:35.829295 (+ 3823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:35.830206 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:35.830218 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:35.830227 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:35.830423 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:35.831000 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:35.831005 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:35.831009 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:35.831565 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:35.831790 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:35.832724 (+ 934us) server_negotiation.cc:299] Negotiation successful
0925 09:29:35.833113 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13695,"thread_start_us":13476,"threads_started":1}
I20260925 09:29:35.834326 23780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:35.789180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39417 (local address 127.10.114.65:56765)
0925 09:29:35.805456 (+ 16276us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:35.805512 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:35.809135 (+ 3623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:35.811912 (+ 2777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:35.811923 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:35.813190 (+ 1267us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:35.814063 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:35.814091 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:35.816785 (+ 2694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:35.816792 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:35.819148 (+ 2356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:35.819166 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:35.821153 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:35.821163 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:35.821466 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:35.822323 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:35.822366 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:35.825138 (+ 2772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:35.829445 (+ 4307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:35.829453 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:35.829469 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:35.830043 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:35.830577 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:35.830583 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:35.830587 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:35.830857 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:35.831761 (+ 904us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:35.831787 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:35.832178 (+ 391us) client_negotiation.cc:769] Sending connection context
0925 09:29:35.832477 (+ 299us) client_negotiation.cc:240] Negotiation successful
0925 09:29:35.833014 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15965,"thread_start_us":15307,"threads_started":1}
I20260925 09:29:35.837934 23778 heartbeater.cc:347] Connected to a master server at 127.10.114.126:39417
I20260925 09:29:35.838255 23778 heartbeater.cc:464] Registering TS with master...
I20260925 09:29:35.839033 23778 heartbeater.cc:511] Master 127.10.114.126:39417 requested a full tablet report, sending...
W20260925 09:29:35.842300 23778 heartbeater.cc:644] Failed to heartbeat to 127.10.114.126:39417 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 09:29:35.842525 23778 heartbeater.cc:464] Registering TS with master...
I20260925 09:29:35.842778 23778 heartbeater.cc:511] Master 127.10.114.126:39417 requested a full tablet report, sending...
I20260925 09:29:35.843981 23778 heartbeater.cc:464] Registering TS with master...
I20260925 09:29:35.844228 23778 heartbeater.cc:511] Master 127.10.114.126:39417 requested a full tablet report, sending...
W20260925 09:29:35.845369 23778 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 09:29:36.164199 23782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:36.164469 23782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:36.164616 23782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:36.174289 23782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:36.174455 23782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:36.174651 23782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:29:36.184818 23782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.114.84:56133/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:46743
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=37391
--tserver_master_addrs=127.10.114.126:39417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:36.186098 23782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:36.187397 23782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:36.201216 23797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:36.201874 23799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:36.201905 23796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:36.202250 23782 server_base.cc:1034] running on GCE node
I20260925 09:29:36.202844 23782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:36.203992 23782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:36.205199 23782 hybrid_clock.cc:648] HybridClock initialized: now 1790328576205145 us; error 54 us; skew 500 ppm
I20260925 09:29:36.205545 23782 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:36 dist-test-slave-9pj8 krb5kdc[22570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328576, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:36.213311 23782 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:29:36.215158 23782 webserver.cc:460] Webserver started at http://127.10.114.66:37391/ using document root <none> and password file <none>
I20260925 09:29:36.215915 23782 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:36.216084 23782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 09:29:36 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328576, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
I20260925 09:29:36.234506 23782 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 09:29:36.256338 23807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:36.258174 23782 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.013s sys 0.007s
I20260925 09:29:36.258415 23782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "e515c0373e714c40965bc88c90fa41a3"
format_stamp: "Formatted at 2026-09-25 09:29:33 on dist-test-slave-9pj8"
server_key: "7c0b04a63f9e0aef93515d8e195a750e"
server_key_iv: "6120e671f27153a7210cb97a5d0a3d7e"
server_key_version: "kuduclusterkey@0"
I20260925 09:29:36.259974 23782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:36.293902 23782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:36.297976 23782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:36.298272 23782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:36.299342 23782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:29:36.300809 23782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:29:36.300921 23782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:36.301066 23782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:29:36.301148 23782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 09:29:36 dist-test-slave-9pj8 krb5kdc[22570](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790328576, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:36.360194 23782 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:46743
I20260925 09:29:36.360237 23921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:46743 every 8 connection(s)
I20260925 09:29:36.361588 23782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:29:36.366267 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 23782
I20260925 09:29:36.367035 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.114.84:56133/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:35235
--local_ip_for_outbound_sockets=127.10.114.67
--tserver_master_addrs=127.10.114.126:39417
--webserver_port=44849
--webserver_interface=127.10.114.67
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.114.84:56133/kms with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:29:36 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328576, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:29:36.415313 23924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:36.367453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39417 (local address 127.10.114.66:48311)
0925 09:29:36.380351 (+ 12898us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:36.380403 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:36.382881 (+ 2478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:36.389046 (+ 6165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:36.389057 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:36.389696 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:36.390639 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:36.390667 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:36.395898 (+ 5231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:36.395907 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:36.398155 (+ 2248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:36.398173 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:36.399471 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:36.399480 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:36.399826 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:36.400587 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:36.400624 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:36.403531 (+ 2907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:36.410595 (+ 7064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:36.410605 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:36.410639 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:36.411212 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:36.411693 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:36.411700 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:36.411704 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:36.412012 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:36.412680 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:36.412688 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:36.413119 (+ 431us) client_negotiation.cc:769] Sending connection context
0925 09:29:36.413421 (+ 302us) client_negotiation.cc:240] Negotiation successful
0925 09:29:36.413972 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12575,"thread_start_us":12361,"threads_started":1}
I20260925 09:29:36.416750 23925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:36.368060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:48311 (local address 127.10.114.126:39417)
0925 09:29:36.388477 (+ 20417us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:36.388484 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:36.388505 (+ 21us) server_negotiation.cc:407] Connection header received
0925 09:29:36.388580 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:36.388585 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:36.388661 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:36.388777 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:36.390873 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:36.392105 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:36.398394 (+ 6289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:36.399164 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:36.400268 (+ 1104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:36.403841 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:36.403849 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:36.403856 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:36.403900 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:36.410440 (+ 6540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:36.411382 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:36.411387 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:36.411391 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:36.411551 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:36.412168 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:36.412172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:36.412175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:36.412523 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:36.416075 (+ 3552us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:36.416214 (+ 139us) server_negotiation.cc:299] Negotiation successful
0925 09:29:36.416419 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20263,"thread_start_us":20843,"threads_started":1}
I20260925 09:29:36.417894 23922 heartbeater.cc:347] Connected to a master server at 127.10.114.126:39417
I20260925 09:29:36.418247 23922 heartbeater.cc:464] Registering TS with master...
I20260925 09:29:36.419138 23922 heartbeater.cc:511] Master 127.10.114.126:39417 requested a full tablet report, sending...
I20260925 09:29:36.421243 23581 ts_manager.cc:194] Registered new tserver with Master: e515c0373e714c40965bc88c90fa41a3 (127.10.114.66:46743)
I20260925 09:29:36.423383 23581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:48311
Sep 25 09:29:36 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328576, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
W20260925 09:29:36.836892 23926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:36.837186 23926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:36.837342 23926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:36.844659 23926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:36.844784 23926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:36.845016 23926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:29:36.845499 23778 heartbeater.cc:464] Registering TS with master...
I20260925 09:29:36.845877 23778 heartbeater.cc:511] Master 127.10.114.126:39417 requested a full tablet report, sending...
I20260925 09:29:36.847060 23581 ts_manager.cc:194] Registered new tserver with Master: 1a668d7b33314865ae897e0f8c16b011 (127.10.114.65:46785)
I20260925 09:29:36.848210 23581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:56765
I20260925 09:29:36.856196 23926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.114.84:56133/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:35235
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=44849
--tserver_master_addrs=127.10.114.126:39417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:36.857434 23926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:36.858741 23926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:36.876365 23932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:36.877046 23933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:36.885663 23926 server_base.cc:1034] running on GCE node
W20260925 09:29:36.885895 23935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:36.886762 23926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:36.888018 23926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:36.889294 23926 hybrid_clock.cc:648] HybridClock initialized: now 1790328576889240 us; error 34 us; skew 500 ppm
I20260925 09:29:36.889648 23926 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:36 dist-test-slave-9pj8 krb5kdc[22570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328576, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:36.893838 23926 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:29:36.896005 23926 webserver.cc:460] Webserver started at http://127.10.114.67:44849/ using document root <none> and password file <none>
I20260925 09:29:36.897032 23926 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:36.897272 23926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 09:29:36 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328576, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
I20260925 09:29:36.933403 23926 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.004s sys 0.008s
I20260925 09:29:36.936491 23943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:36.938055 23926 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:29:36.938230 23926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "f3061813deb64a3b8ad2435986327438"
format_stamp: "Formatted at 2026-09-25 09:29:33 on dist-test-slave-9pj8"
server_key: "2193ba8cb6e68c9ff840d56a738b4dd7"
server_key_iv: "7049845470c8b3ef6fdcb48b0564f259"
server_key_version: "kuduclusterkey@0"
I20260925 09:29:36.939188 23926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:36.952016 23926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:36.955828 23926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:36.956092 23926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:36.957046 23926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:29:36.958492 23926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:29:36.958599 23926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:36.958716 23926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:29:36.958806 23926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:37.011808 23926 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:35235
I20260925 09:29:37.011852 24055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:35235 every 8 connection(s)
I20260925 09:29:37.013319 23926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:29:37.019008 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 23926
Sep 25 09:29:37 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328576, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:29:37.060870 24058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.020142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35282 (local address 127.10.114.126:39417)
0925 09:29:37.040082 (+ 19940us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:37.040091 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:37.040116 (+ 25us) server_negotiation.cc:407] Connection header received
0925 09:29:37.040196 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:37.040204 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:37.040322 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:37.040454 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:37.044477 (+ 4023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.045707 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.047737 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.048393 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.048508 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:37.052030 (+ 3522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:37.052036 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:37.052041 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:37.052081 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:37.055962 (+ 3881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:37.057532 (+ 1570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:37.057537 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:37.057541 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:37.057670 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:37.058129 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:37.058136 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:37.058142 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:37.058522 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:37.058769 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:37.060142 (+ 1373us) server_negotiation.cc:299] Negotiation successful
0925 09:29:37.060445 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19753,"thread_start_us":18521,"threads_started":1}
I20260925 09:29:37.066170 24060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.016610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:53185 (local address 127.10.114.126:39417)
0925 09:29:37.038637 (+ 22027us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:37.038647 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:37.043768 (+ 5121us) server_negotiation.cc:407] Connection header received
0925 09:29:37.046591 (+ 2823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:37.046596 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:37.046679 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:37.046752 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:37.049270 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.050524 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.053017 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.053721 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.053835 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:37.059062 (+ 5227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:37.059068 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:37.059073 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:37.059105 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:37.062626 (+ 3521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:37.063499 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:37.063504 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:37.063507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:37.063647 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:37.064180 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:37.064185 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:37.064188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:37.064532 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:37.064694 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:37.065504 (+ 810us) server_negotiation.cc:299] Negotiation successful
0925 09:29:37.065784 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21869,"thread_start_us":27199,"threads_started":1}
I20260925 09:29:37.068508 24059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.017302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39417 (local address 127.10.114.67:53185)
0925 09:29:37.043624 (+ 26322us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:37.043670 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:37.046289 (+ 2619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:37.047490 (+ 1201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:37.047519 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:37.048178 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:37.049033 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:37.049059 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.050730 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.050736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:37.052856 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:37.052873 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.054203 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.054212 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:37.054513 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:37.055197 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:37.055226 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:37.058779 (+ 3553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:37.062789 (+ 4010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:37.062797 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:37.062812 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:37.063356 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:37.063819 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:37.063835 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:37.063839 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:37.064060 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:37.064661 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:37.064668 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:37.065071 (+ 403us) client_negotiation.cc:769] Sending connection context
0925 09:29:37.067000 (+ 1929us) client_negotiation.cc:240] Negotiation successful
0925 09:29:37.067519 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26007,"thread_start_us":25817,"threads_started":1}
I20260925 09:29:37.070020 24056 heartbeater.cc:347] Connected to a master server at 127.10.114.126:39417
I20260925 09:29:37.070353 24056 heartbeater.cc:464] Registering TS with master...
I20260925 09:29:37.071211 24056 heartbeater.cc:511] Master 127.10.114.126:39417 requested a full tablet report, sending...
I20260925 09:29:37.072721 23581 ts_manager.cc:194] Registered new tserver with Master: f3061813deb64a3b8ad2435986327438 (127.10.114.67:35235)
I20260925 09:29:37.073776 23581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:53185
I20260925 09:29:37.078850 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 09:29:37 dist-test-slave-9pj8 krb5kdc[22570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328576, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for HTTP/127.10.114.84@KRBTEST.COM
I20260925 09:29:37.111706 24060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.094114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35294 (local address 127.10.114.126:39417)
0925 09:29:37.094311 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:37.094319 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:37.094683 (+ 364us) server_negotiation.cc:407] Connection header received
0925 09:29:37.095444 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:37.095448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:37.095522 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:37.095578 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:37.097636 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.098625 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.100610 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.101276 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.101379 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:37.105043 (+ 3664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:37.105050 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:37.105055 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:37.105092 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:37.108532 (+ 3440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:37.109360 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:37.109366 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:37.109369 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:37.109504 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:37.109941 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:37.109947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:37.109950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:37.110318 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:37.110530 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:37.111088 (+ 558us) server_negotiation.cc:299] Negotiation successful
0925 09:29:37.111347 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 09:29:37.118180 23581 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:29:37.129658 23581 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:29:37.197237 24073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.169377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:46785 (local address 127.0.0.1:40758)
0925 09:29:37.187269 (+ 17892us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:37.187318 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:37.187530 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:37.188527 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:37.188534 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:37.188560 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:29:37.188976 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:37.188991 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.191531 (+ 2540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.191536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:37.193625 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:37.193642 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.196260 (+ 2618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.196265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:37.196451 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:37.196503 (+ 52us) client_negotiation.cc:769] Sending connection context
0925 09:29:37.196636 (+ 133us) client_negotiation.cc:240] Negotiation successful
0925 09:29:37.196825 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17567,"mutex_wait_us":25,"thread_start_us":18457,"threads_started":1}
I20260925 09:29:37.197997 24072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.169606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:46743 (local address 127.0.0.1:38574)
0925 09:29:37.188066 (+ 18460us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:37.188083 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:37.188199 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:37.189103 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:37.189107 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:37.189133 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:29:37.189455 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:37.189466 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.191288 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.191293 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:37.194157 (+ 2864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:37.194169 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.196183 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.196191 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:37.197396 (+ 1205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:37.197428 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 09:29:37.197518 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 09:29:37.197641 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18346,"mutex_wait_us":1484,"thread_start_us":16754,"threads_started":1}
I20260925 09:29:37.198240 24075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.170033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40758 (local address 127.10.114.65:46785)
0925 09:29:37.176242 (+ 6209us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:37.176252 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:37.187429 (+ 11177us) server_negotiation.cc:407] Connection header received
0925 09:29:37.187757 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:37.187793 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:37.188081 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:37.188267 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:29:37.189245 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.191388 (+ 2143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.193798 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.196066 (+ 2268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.196222 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:37.196508 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:37.197726 (+ 1218us) server_negotiation.cc:299] Negotiation successful
0925 09:29:37.197913 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6030,"thread_start_us":5863,"threads_started":1}
I20260925 09:29:37.198665 24074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.170063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38574 (local address 127.10.114.66:46743)
0925 09:29:37.186857 (+ 16794us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:37.186865 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:37.188326 (+ 1461us) server_negotiation.cc:407] Connection header received
0925 09:29:37.188438 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:37.188445 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:37.188671 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:37.188856 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:29:37.189615 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.191102 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.194322 (+ 3220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.195665 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.195789 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:37.195986 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:37.198195 (+ 2209us) server_negotiation.cc:299] Negotiation successful
0925 09:29:37.198374 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16625,"thread_start_us":22472,"threads_started":1}
I20260925 09:29:37.203261 23713 tablet_service.cc:1467] Processing CreateTablet for tablet 5d88e18ce1f845e9b04f65a9d7aaea3d (DEFAULT_TABLE table=test-table [id=619040e331fc42699568b4c91f259d87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:29:37.205250 23713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d88e18ce1f845e9b04f65a9d7aaea3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:29:37.212709 24073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.193802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:35235 (local address 127.0.0.1:51682)
0925 09:29:37.197454 (+ 3652us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:37.197471 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:37.197548 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:37.201104 (+ 3556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:37.201109 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:37.201134 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:29:37.201319 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:37.201329 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.203129 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.203154 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:37.205140 (+ 1986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:37.205168 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.211978 (+ 6810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.211985 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:37.212174 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:37.212212 (+ 38us) client_negotiation.cc:769] Sending connection context
0925 09:29:37.212279 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 09:29:37.212431 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3575,"thread_start_us":15079,"threads_started":1}
I20260925 09:29:37.212504 23857 tablet_service.cc:1467] Processing CreateTablet for tablet 5d88e18ce1f845e9b04f65a9d7aaea3d (DEFAULT_TABLE table=test-table [id=619040e331fc42699568b4c91f259d87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:29:37.213708 23857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d88e18ce1f845e9b04f65a9d7aaea3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:29:37.213816 24076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.187886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51682 (local address 127.10.114.67:35235)
0925 09:29:37.199327 (+ 11441us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:37.199335 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:37.199365 (+ 30us) server_negotiation.cc:407] Connection header received
0925 09:29:37.199447 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:37.199454 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:37.199666 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:37.201124 (+ 1458us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:29:37.201437 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.203003 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.210312 (+ 7309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.211759 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.213040 (+ 1281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:37.213263 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:37.213331 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 09:29:37.213478 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11270,"thread_start_us":11111,"threads_started":1}
I20260925 09:29:37.217242 23991 tablet_service.cc:1467] Processing CreateTablet for tablet 5d88e18ce1f845e9b04f65a9d7aaea3d (DEFAULT_TABLE table=test-table [id=619040e331fc42699568b4c91f259d87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:29:37.218914 23991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d88e18ce1f845e9b04f65a9d7aaea3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:29:37.249388 24078 tablet_bootstrap.cc:492] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3: Bootstrap starting.
I20260925 09:29:37.254375 24079 tablet_bootstrap.cc:492] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011: Bootstrap starting.
I20260925 09:29:37.258563 24079 tablet_bootstrap.cc:654] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011: Neither blocks nor log segments found. Creating new log.
I20260925 09:29:37.259892 24079 log.cc:824] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011: Log is configured to *not* fsync() on all Append() calls
I20260925 09:29:37.263079 24078 tablet_bootstrap.cc:654] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3: Neither blocks nor log segments found. Creating new log.
I20260925 09:29:37.265522 24080 tablet_bootstrap.cc:492] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438: Bootstrap starting.
I20260925 09:29:37.268971 24079 tablet_bootstrap.cc:492] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011: No bootstrap required, opened a new log
I20260925 09:29:37.269233 24079 ts_tablet_manager.cc:1397] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260925 09:29:37.264657 24078 log.cc:824] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3: Log is configured to *not* fsync() on all Append() calls
I20260925 09:29:37.279814 24079 raft_consensus.cc:348] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a668d7b33314865ae897e0f8c16b011" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46785 } } peers { permanent_uuid: "e515c0373e714c40965bc88c90fa41a3" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46743 } } peers { permanent_uuid: "f3061813deb64a3b8ad2435986327438" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 35235 } }
I20260925 09:29:37.280251 24079 raft_consensus.cc:374] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:29:37.280337 24079 raft_consensus.cc:729] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a668d7b33314865ae897e0f8c16b011, State: Initialized, Role: FOLLOWER
I20260925 09:29:37.281010 24079 consensus_queue.cc:260] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a668d7b33314865ae897e0f8c16b011" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46785 } } peers { permanent_uuid: "e515c0373e714c40965bc88c90fa41a3" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46743 } } peers { permanent_uuid: "f3061813deb64a3b8ad2435986327438" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 35235 } }
I20260925 09:29:37.284008 24080 tablet_bootstrap.cc:654] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438: Neither blocks nor log segments found. Creating new log.
I20260925 09:29:37.289258 24079 ts_tablet_manager.cc:1428] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011: Time spent starting tablet: real 0.020s user 0.019s sys 0.000s
I20260925 09:29:37.289592 23778 heartbeater.cc:503] Master 127.10.114.126:39417 was elected leader, sending a full tablet report...
I20260925 09:29:37.290390 24080 log.cc:824] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438: Log is configured to *not* fsync() on all Append() calls
I20260925 09:29:37.296267 24078 tablet_bootstrap.cc:492] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3: No bootstrap required, opened a new log
I20260925 09:29:37.296543 24078 ts_tablet_manager.cc:1397] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3: Time spent bootstrapping tablet: real 0.047s user 0.008s sys 0.000s
I20260925 09:29:37.309317 24078 raft_consensus.cc:348] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a668d7b33314865ae897e0f8c16b011" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46785 } } peers { permanent_uuid: "e515c0373e714c40965bc88c90fa41a3" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46743 } } peers { permanent_uuid: "f3061813deb64a3b8ad2435986327438" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 35235 } }
I20260925 09:29:37.309751 24078 raft_consensus.cc:374] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:29:37.309840 24078 raft_consensus.cc:729] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e515c0373e714c40965bc88c90fa41a3, State: Initialized, Role: FOLLOWER
I20260925 09:29:37.310552 24078 consensus_queue.cc:260] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a668d7b33314865ae897e0f8c16b011" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46785 } } peers { permanent_uuid: "e515c0373e714c40965bc88c90fa41a3" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46743 } } peers { permanent_uuid: "f3061813deb64a3b8ad2435986327438" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 35235 } }
I20260925 09:29:37.313915 24080 tablet_bootstrap.cc:492] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438: No bootstrap required, opened a new log
I20260925 09:29:37.314340 24080 ts_tablet_manager.cc:1397] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438: Time spent bootstrapping tablet: real 0.049s user 0.005s sys 0.011s
W20260925 09:29:37.314833 23779 tablet.cc:2369] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:29:37.320472 23922 heartbeater.cc:503] Master 127.10.114.126:39417 was elected leader, sending a full tablet report...
I20260925 09:29:37.321195 24078 ts_tablet_manager.cc:1428] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3: Time spent starting tablet: real 0.025s user 0.020s sys 0.000s
I20260925 09:29:37.327536 24080 raft_consensus.cc:348] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a668d7b33314865ae897e0f8c16b011" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46785 } } peers { permanent_uuid: "e515c0373e714c40965bc88c90fa41a3" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46743 } } peers { permanent_uuid: "f3061813deb64a3b8ad2435986327438" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 35235 } }
I20260925 09:29:37.328058 24080 raft_consensus.cc:374] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:29:37.328218 24080 raft_consensus.cc:729] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3061813deb64a3b8ad2435986327438, State: Initialized, Role: FOLLOWER
I20260925 09:29:37.328971 24080 consensus_queue.cc:260] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a668d7b33314865ae897e0f8c16b011" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46785 } } peers { permanent_uuid: "e515c0373e714c40965bc88c90fa41a3" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46743 } } peers { permanent_uuid: "f3061813deb64a3b8ad2435986327438" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 35235 } }
I20260925 09:29:37.333089 24056 heartbeater.cc:503] Master 127.10.114.126:39417 was elected leader, sending a full tablet report...
I20260925 09:29:37.333832 24080 ts_tablet_manager.cc:1428] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438: Time spent starting tablet: real 0.019s user 0.016s sys 0.000s
I20260925 09:29:37.388448 24082 raft_consensus.cc:482] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:29:37.388746 24082 raft_consensus.cc:504] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a668d7b33314865ae897e0f8c16b011" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46785 } } peers { permanent_uuid: "e515c0373e714c40965bc88c90fa41a3" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46743 } } peers { permanent_uuid: "f3061813deb64a3b8ad2435986327438" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 35235 } }
I20260925 09:29:37.390448 24082 leader_election.cc:288] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e515c0373e714c40965bc88c90fa41a3 (127.10.114.66:46743), f3061813deb64a3b8ad2435986327438 (127.10.114.67:35235)
I20260925 09:29:37.398921 24076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.392166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:48115 (local address 127.10.114.67:35235)
0925 09:29:37.392343 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:37.392350 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:37.392373 (+ 23us) server_negotiation.cc:407] Connection header received
0925 09:29:37.392659 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:37.392664 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:37.392749 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:37.392831 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:29:37.393568 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.394290 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.396783 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.397966 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.398138 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:37.398214 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:37.398552 (+ 338us) server_negotiation.cc:299] Negotiation successful
0925 09:29:37.398678 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 09:29:37.399138 24074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.391369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:58519 (local address 127.10.114.66:46743)
0925 09:29:37.391615 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:37.391621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:37.392141 (+ 520us) server_negotiation.cc:407] Connection header received
0925 09:29:37.392695 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:37.392700 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:37.392765 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:37.392824 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:29:37.393639 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.394666 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.397256 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.398054 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.398205 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:37.398301 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:37.398751 (+ 450us) server_negotiation.cc:299] Negotiation successful
0925 09:29:37.398889 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260925 09:29:37.399312 24094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.391641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:35235 (local address 127.10.114.65:48115)
0925 09:29:37.392265 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:37.392284 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:37.392421 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:37.392923 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:37.392928 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:37.392954 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:29:37.393385 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:37.393399 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.394414 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.394421 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:37.396607 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:37.396622 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.398186 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.398192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:37.398373 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:37.398421 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 09:29:37.398508 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 09:29:37.398627 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":66,"thread_start_us":582,"threads_started":1}
I20260925 09:29:37.399904 24093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.390840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:46743 (local address 127.10.114.65:58519)
0925 09:29:37.391999 (+ 1159us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:37.392044 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:37.392467 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:37.393022 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:37.393028 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:37.393069 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:29:37.393429 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:37.393438 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.394822 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.394828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:37.397114 (+ 2286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:37.397123 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.398434 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.398439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:37.398589 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:37.398618 (+ 29us) client_negotiation.cc:769] Sending connection context
0925 09:29:37.399366 (+ 748us) client_negotiation.cc:240] Negotiation successful
0925 09:29:37.399505 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":37,"thread_start_us":358,"threads_started":1}
I20260925 09:29:37.400319 24011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d88e18ce1f845e9b04f65a9d7aaea3d" candidate_uuid: "1a668d7b33314865ae897e0f8c16b011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3061813deb64a3b8ad2435986327438" is_pre_election: true
I20260925 09:29:37.400868 24011 raft_consensus.cc:2454] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a668d7b33314865ae897e0f8c16b011 in term 0.
I20260925 09:29:37.400812 23877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d88e18ce1f845e9b04f65a9d7aaea3d" candidate_uuid: "1a668d7b33314865ae897e0f8c16b011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e515c0373e714c40965bc88c90fa41a3" is_pre_election: true
I20260925 09:29:37.401294 23877 raft_consensus.cc:2454] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a668d7b33314865ae897e0f8c16b011 in term 0.
I20260925 09:29:37.401530 23668 leader_election.cc:302] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a668d7b33314865ae897e0f8c16b011, f3061813deb64a3b8ad2435986327438; no voters:
I20260925 09:29:37.401917 24082 raft_consensus.cc:2781] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:29:37.402083 24082 raft_consensus.cc:482] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:29:37.402206 24082 raft_consensus.cc:3037] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:29:37.404433 24082 raft_consensus.cc:504] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a668d7b33314865ae897e0f8c16b011" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46785 } } peers { permanent_uuid: "e515c0373e714c40965bc88c90fa41a3" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46743 } } peers { permanent_uuid: "f3061813deb64a3b8ad2435986327438" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 35235 } }
I20260925 09:29:37.404927 24082 leader_election.cc:288] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [CANDIDATE]: Term 1 election: Requested vote from peers e515c0373e714c40965bc88c90fa41a3 (127.10.114.66:46743), f3061813deb64a3b8ad2435986327438 (127.10.114.67:35235)
I20260925 09:29:37.405458 24011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d88e18ce1f845e9b04f65a9d7aaea3d" candidate_uuid: "1a668d7b33314865ae897e0f8c16b011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3061813deb64a3b8ad2435986327438"
I20260925 09:29:37.405664 24011 raft_consensus.cc:3037] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:29:37.407974 24011 raft_consensus.cc:2454] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a668d7b33314865ae897e0f8c16b011 in term 1.
I20260925 09:29:37.408003 23877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d88e18ce1f845e9b04f65a9d7aaea3d" candidate_uuid: "1a668d7b33314865ae897e0f8c16b011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e515c0373e714c40965bc88c90fa41a3"
I20260925 09:29:37.408187 23877 raft_consensus.cc:3037] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:29:37.408375 23668 leader_election.cc:302] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a668d7b33314865ae897e0f8c16b011, f3061813deb64a3b8ad2435986327438; no voters:
I20260925 09:29:37.408679 24082 raft_consensus.cc:2781] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [term 1 FOLLOWER]: Leader election won for term 1
W20260925 09:29:37.409032 23923 tablet.cc:2369] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:29:37.410887 23877 raft_consensus.cc:2454] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a668d7b33314865ae897e0f8c16b011 in term 1.
I20260925 09:29:37.411367 24082 raft_consensus.cc:686] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [term 1 LEADER]: Becoming Leader. State: Replica: 1a668d7b33314865ae897e0f8c16b011, State: Running, Role: LEADER
I20260925 09:29:37.411980 24082 consensus_queue.cc:237] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a668d7b33314865ae897e0f8c16b011" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46785 } } peers { permanent_uuid: "e515c0373e714c40965bc88c90fa41a3" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46743 } } peers { permanent_uuid: "f3061813deb64a3b8ad2435986327438" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 35235 } }
I20260925 09:29:37.417173 23581 catalog_manager.cc:5495] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a668d7b33314865ae897e0f8c16b011 (127.10.114.65). New cstate: current_term: 1 leader_uuid: "1a668d7b33314865ae897e0f8c16b011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a668d7b33314865ae897e0f8c16b011" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e515c0373e714c40965bc88c90fa41a3" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3061813deb64a3b8ad2435986327438" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 35235 } health_report { overall_health: UNKNOWN } } }
I20260925 09:29:37.455762 24075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.449732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40766 (local address 127.10.114.65:46785)
0925 09:29:37.449914 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:37.449920 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:37.449962 (+ 42us) server_negotiation.cc:407] Connection header received
0925 09:29:37.450206 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:37.450211 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:37.450319 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:37.450471 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:29:37.450891 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.452069 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.453413 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.453998 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.454122 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:37.454569 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:29:37.455220 (+ 651us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:29:37.455377 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:37.455423 (+ 46us) server_negotiation.cc:299] Negotiation successful
0925 09:29:37.455498 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:29:37.465009 23877 raft_consensus.cc:1261] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3 [term 1 FOLLOWER]: Refusing update from remote peer 1a668d7b33314865ae897e0f8c16b011: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:29:37.465098 24011 raft_consensus.cc:1261] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438 [term 1 FOLLOWER]: Refusing update from remote peer 1a668d7b33314865ae897e0f8c16b011: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:29:37.466104 24096 consensus_queue.cc:1035] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3061813deb64a3b8ad2435986327438" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 35235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:29:37.466521 24082 consensus_queue.cc:1035] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e515c0373e714c40965bc88c90fa41a3" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 46743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:29:37.485569 24101 mvcc.cc:205] Tried to move back new op lower bound from 7333185853285728256 to 7333185853083799552. Current Snapshot: MvccSnapshot[applied={T|T < 7333185853285728256}]
I20260925 09:29:37.510566 23581 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:29:37.510828 23581 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:29:37.514773 23581 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 31d38d18783c4b289f8ff541b564aa0b: Sending DeleteTablet for 3 replicas of tablet 5d88e18ce1f845e9b04f65a9d7aaea3d
I20260925 09:29:37.516136 23991 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d88e18ce1f845e9b04f65a9d7aaea3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:29:37 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:51682
I20260925 09:29:37.516255 23713 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d88e18ce1f845e9b04f65a9d7aaea3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:29:37 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:40758
I20260925 09:29:37.516947 23857 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d88e18ce1f845e9b04f65a9d7aaea3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:29:37 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:38574
I20260925 09:29:37.525143 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 23618
I20260925 09:29:37.540409 24111 tablet_replica.cc:330] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3: stopping tablet replica
I20260925 09:29:37.541076 24111 raft_consensus.cc:2229] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:29:37.541818 24110 tablet_replica.cc:330] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011: stopping tablet replica
I20260925 09:29:37.542562 24111 raft_consensus.cc:2258] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:29:37.543174 24110 raft_consensus.cc:2229] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:29:37.544580 24110 raft_consensus.cc:2258] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:29:37.546747 24111 ts_tablet_manager.cc:1905] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:29:37.548923 24110 ts_tablet_manager.cc:1905] T 5d88e18ce1f845e9b04f65a9d7aaea3d P 1a668d7b33314865ae897e0f8c16b011: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:29:37.549515 24109 tablet_replica.cc:330] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438: stopping tablet replica
I20260925 09:29:37.549887 24109 raft_consensus.cc:2229] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:29:37.550323 24109 raft_consensus.cc:2258] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:29:37.552217 24109 ts_tablet_manager.cc:1905] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:29:37.557780 24111 ts_tablet_manager.cc:1918] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:29:37.557941 24111 log.cc:1196] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/5d88e18ce1f845e9b04f65a9d7aaea3d
I20260925 09:29:37.558539 24111 ts_tablet_manager.cc:1939] T 5d88e18ce1f845e9b04f65a9d7aaea3d P e515c0373e714c40965bc88c90fa41a3: Deleting consensus metadata
I20260925 09:29:37.558893 24109 ts_tablet_manager.cc:1918] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:29:37.559108 24109 log.cc:1196] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/5d88e18ce1f845e9b04f65a9d7aaea3d
I20260925 09:29:37.559639 24109 ts_tablet_manager.cc:1939] T 5d88e18ce1f845e9b04f65a9d7aaea3d P f3061813deb64a3b8ad2435986327438: Deleting consensus metadata
I20260925 09:29:37.555007 24075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.528935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40778 (local address 127.10.114.65:46785)
0925 09:29:37.529482 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:37.529490 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:37.529514 (+ 24us) server_negotiation.cc:407] Connection header received
0925 09:29:37.529588 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:37.529596 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:37.529687 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:37.529796 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:37.532107 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.533503 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.535991 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.536873 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.537010 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:37.541130 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:37.541144 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:37.541164 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:37.541234 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:37.545498 (+ 4264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:37.546442 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:37.546452 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:37.546455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:37.546615 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:37.552315 (+ 5700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:37.552321 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:37.552325 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:37.552810 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:37.553107 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:37.554325 (+ 1218us) server_negotiation.cc:299] Negotiation successful
0925 09:29:37.554576 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376}
I20260925 09:29:37.560616 23773 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:29:37.561606 23568 catalog_manager.cc:4841] TS f3061813deb64a3b8ad2435986327438 (127.10.114.67:35235): tablet 5d88e18ce1f845e9b04f65a9d7aaea3d (table test-table [id=619040e331fc42699568b4c91f259d87]) successfully deleted
I20260925 09:29:37.561916 23568 catalog_manager.cc:4841] TS e515c0373e714c40965bc88c90fa41a3 (127.10.114.66:46743): tablet 5d88e18ce1f845e9b04f65a9d7aaea3d (table test-table [id=619040e331fc42699568b4c91f259d87]) successfully deleted
I20260925 09:29:37.658830 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 23618
I20260925 09:29:37.678365 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 23782
W20260925 09:29:37.678501 23566 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.114.65:46785 (error 108)
I20260925 09:29:37.680720 24073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.680289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:46785 (local address 127.0.0.1:40788)
0925 09:29:37.680455 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:37.680568 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:46785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260925 09:29:37.681027 23566 catalog_manager.cc:4584] TS 1a668d7b33314865ae897e0f8c16b011 (127.10.114.65:46785): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5d88e18ce1f845e9b04f65a9d7aaea3d: Network error: Client connection negotiation failed: client connection to 127.10.114.65:46785: connect: Connection refused (error 111)
I20260925 09:29:37.693876 24074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.679554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38584 (local address 127.10.114.66:46743)
0925 09:29:37.679737 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:37.679742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:37.679759 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:29:37.679870 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:37.679876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:37.679955 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:37.680038 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:37.681703 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.682420 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.684197 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.684591 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.684712 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:37.687609 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:37.687625 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:37.687643 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:37.687680 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:37.691061 (+ 3381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:37.691753 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:37.691761 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:37.691765 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:37.691945 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:37.692387 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:37.692392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:37.692395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:37.692819 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:37.693079 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:37.693305 (+ 226us) server_negotiation.cc:299] Negotiation successful
0925 09:29:37.693528 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 09:29:37.694557 23917 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:29:37.734860 24073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.734342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:46785 (local address 127.0.0.1:40790)
0925 09:29:37.734577 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:37.734695 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:46785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260925 09:29:37.736099 24073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.735638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:46785 (local address 127.0.0.1:40792)
0925 09:29:37.735825 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:37.735935 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:46785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260925 09:29:37.783627 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 23782
I20260925 09:29:37.796761 24073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.796295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:46785 (local address 127.0.0.1:40804)
0925 09:29:37.796487 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:37.796593 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:46785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260925 09:29:37.797818 24073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.797441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:46785 (local address 127.0.0.1:40816)
0925 09:29:37.797588 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:37.797665 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:46785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 09:29:37.801119 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 23926
I20260925 09:29:37.813836 24076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.801776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51692 (local address 127.10.114.67:35235)
0925 09:29:37.801936 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:37.801940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:37.801963 (+ 23us) server_negotiation.cc:407] Connection header received
0925 09:29:37.802105 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:37.802109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:37.802180 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:37.802270 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:37.803580 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.804383 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.805613 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:37.805968 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:37.806057 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:37.808673 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:37.808686 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:37.808701 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:37.808756 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:37.811308 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:37.812009 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:37.812017 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:37.812019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:37.812149 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:37.812528 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:37.812531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:37.812533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:37.812843 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:37.813038 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:37.813352 (+ 314us) server_negotiation.cc:299] Negotiation successful
0925 09:29:37.813541 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:29:37.814483 24051 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:29:37.878080 24073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.877550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:46785 (local address 127.0.0.1:40826)
0925 09:29:37.877798 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:37.877907 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:46785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260925 09:29:37.879242 24073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:37.878731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:46785 (local address 127.0.0.1:40838)
0925 09:29:37.878978 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:37.879065 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:46785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260925 09:29:37.901530 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 23926
I20260925 09:29:37.918135 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 23548
I20260925 09:29:37.919308 23611 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:29:38.029247 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 23548
I20260925 09:29:38.053076 10697 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 09:29:38.397398 10697 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 09:29:38.397609 10697 mini_ranger.cc:66] Stopping Ranger...
I20260925 09:29:38.741791 10697 mini_ranger.cc:68] Stopped Ranger
2026-09-25 09:29:38.742 UTC [22600] LOG: received smart shutdown request
2026-09-25 09:29:38.744 UTC [22600] LOG: background worker "logical replication launcher" (PID 22609) exited with exit code 1
2026-09-25 09:29:38.745 UTC [22604] LOG: shutting down
2026-09-25 09:29:38.907 UTC [22600] LOG: database system is shut down
2026-09-25T09:29:38Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51413 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:29:39 dist-test-slave-9pj8 krb5kdc[24125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:29:39 dist-test-slave-9pj8 krb5kdc[24125](info): set up 2 sockets
Sep 25 09:29:39 dist-test-slave-9pj8 krb5kdc[24125](info): commencing operation
krb5kdc: starting...
W20260925 09:29:40.120491 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:29:40 dist-test-slave-9pj8 krb5kdc[24125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:29:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:29:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:29:40.327534 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:42719
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:42577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:42719,127.10.114.125:46537,127.10.114.124:36245
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:29:40.640790 24138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:40.641103 24138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:40.641291 24138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:40.648157 24138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:29:40.648316 24138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:40.648427 24138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:40.648519 24138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:29:40.648602 24138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:29:40.657826 24138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:42577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:42719,127.10.114.125:46537,127.10.114.124:36245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:42719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:40.659117 24138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:40.660374 24138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:40.671824 24144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:40.672032 24143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:40.672215 24146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:40.672756 24138 server_base.cc:1034] running on GCE node
I20260925 09:29:40.673451 24138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:40.674832 24138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:40.676088 24138 hybrid_clock.cc:648] HybridClock initialized: now 1790328580676043 us; error 32 us; skew 500 ppm
I20260925 09:29:40.676594 24138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:40 dist-test-slave-9pj8 krb5kdc[24125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328580, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:40.680111 24138 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:29:40.681507 24138 webserver.cc:460] Webserver started at http://127.10.114.126:34049/ using document root <none> and password file <none>
I20260925 09:29:40.682224 24138 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:40.682370 24138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:29:40.682715 24138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:29:40.685781 24138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "ec98e9821deb4c70a9e470dddf611f1b"
format_stamp: "Formatted at 2026-09-25 09:29:40 on dist-test-slave-9pj8"
server_key: "4334ccacc5e9051659956a04bba41d34"
server_key_iv: "c2a14e7a5b4b77908141d82a7175655e"
server_key_version: "encryptionkey@0"
I20260925 09:29:40.686617 24138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "ec98e9821deb4c70a9e470dddf611f1b"
format_stamp: "Formatted at 2026-09-25 09:29:40 on dist-test-slave-9pj8"
server_key: "4334ccacc5e9051659956a04bba41d34"
server_key_iv: "c2a14e7a5b4b77908141d82a7175655e"
server_key_version: "encryptionkey@0"
I20260925 09:29:40.691673 24138 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260925 09:29:40.694914 24153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:40.696336 24138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260925 09:29:40.696520 24138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "ec98e9821deb4c70a9e470dddf611f1b"
format_stamp: "Formatted at 2026-09-25 09:29:40 on dist-test-slave-9pj8"
server_key: "4334ccacc5e9051659956a04bba41d34"
server_key_iv: "c2a14e7a5b4b77908141d82a7175655e"
server_key_version: "encryptionkey@0"
I20260925 09:29:40.696702 24138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:40.724874 24138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:40.729358 24138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:40.729648 24138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:40.747833 24138 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:42719
I20260925 09:29:40.747900 24204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:42719 every 8 connection(s)
I20260925 09:29:40.749099 24138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:29:40.752650 24205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:29:40.756785 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 24138
I20260925 09:29:40.757002 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:29:40.757619 10697 external_mini_cluster.cc:1428] Setting key 691ee686efc32f3c73bf402e918e371e
I20260925 09:29:40.759060 24205 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:29:40 dist-test-slave-9pj8 krb5kdc[24125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328580, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.114.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.125@KRBTEST.COM" created.
I20260925 09:29:40.783047 24209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:40.763212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41996 (local address 127.10.114.126:42719)
0925 09:29:40.764358 (+ 1146us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:40.764365 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:40.764415 (+ 50us) server_negotiation.cc:407] Connection header received
0925 09:29:40.765913 (+ 1498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:40.765919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:40.766131 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:40.766292 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:40.767909 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:40.769010 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:40.771158 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:40.771698 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:40.771908 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:40.776378 (+ 4470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:40.776386 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:40.776397 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:40.776425 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:40.779182 (+ 2757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:40.779898 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:40.779904 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:40.779906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:40.780042 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:40.780386 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:40.780389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:40.780391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:40.780901 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:40.781125 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:40.781722 (+ 597us) server_negotiation.cc:299] Negotiation successful
0925 09:29:40.782007 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":934,"thread_start_us":758,"threads_started":1}
I20260925 09:29:40.783074 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:40.761958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42719 (local address 127.0.0.1:41996)
0925 09:29:40.762756 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:40.762802 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:40.765518 (+ 2716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:40.766383 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:40.766392 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:40.767174 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:40.767663 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:40.767689 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:40.769159 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:40.769166 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:40.771015 (+ 1849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:40.771032 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:40.772022 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:40.772032 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:40.772248 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:40.772958 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:40.772991 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:40.776113 (+ 3122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:40.779298 (+ 3185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:40.779308 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:40.779317 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:40.779726 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:40.780137 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:40.780141 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:40.780143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:40.780309 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:40.781065 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:40.781073 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:40.781307 (+ 234us) client_negotiation.cc:769] Sending connection context
0925 09:29:40.781588 (+ 281us) client_negotiation.cc:240] Negotiation successful
0925 09:29:40.782206 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"mutex_wait_us":86,"thread_start_us":342,"threads_started":1}
I20260925 09:29:40.786218 24205 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:29:40.787173 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:40.786851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:46537 (local address 127.0.0.1:38556)
0925 09:29:40.786988 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:40.787081 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:46537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 09:29:40.787392 24154 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.114.125:46537: connect: Connection refused (error 111)
I20260925 09:29:40.788861 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:40.788554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:46537 (local address 127.0.0.1:38564)
0925 09:29:40.788684 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:40.788744 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:46537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260925 09:29:40.789143 24205 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.125:46537: Network error: Client connection negotiation failed: client connection to 127.10.114.125:46537: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.125.keytab.
Entry for principal kudu/127.10.114.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.125.keytab.
I20260925 09:29:40.808627 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.125:46537
--webserver_interface=127.10.114.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.125.keytab
--principal=kudu/127.10.114.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:42577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:42719,127.10.114.125:46537,127.10.114.124:36245
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:29:40.838524 24205 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } attempt: 1
I20260925 09:29:40.839813 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:40.839310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:46537 (local address 127.0.0.1:38568)
0925 09:29:40.839511 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:40.839634 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:46537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260925 09:29:40.840941 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:40.840604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:46537 (local address 127.0.0.1:38574)
0925 09:29:40.840745 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:40.840809 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:46537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260925 09:29:40.841217 24205 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.125:46537: Network error: Client connection negotiation failed: client connection to 127.10.114.125:46537: connect: Connection refused (error 111)
I20260925 09:29:40.909482 24205 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } attempt: 2
I20260925 09:29:40.910686 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:40.910242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:46537 (local address 127.0.0.1:38586)
0925 09:29:40.910468 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:40.910559 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:46537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260925 09:29:40.911743 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:40.911424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:46537 (local address 127.0.0.1:38588)
0925 09:29:40.911566 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:40.911628 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:46537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 09:29:40.912012 24205 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.125:46537: Network error: Client connection negotiation failed: client connection to 127.10.114.125:46537: connect: Connection refused (error 111)
I20260925 09:29:41.003290 24205 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } attempt: 3
I20260925 09:29:41.004635 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.004168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:46537 (local address 127.0.0.1:38596)
0925 09:29:41.004381 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.004500 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:46537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260925 09:29:41.005718 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.005368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:46537 (local address 127.0.0.1:38602)
0925 09:29:41.005504 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.005575 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:46537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 09:29:41.005970 24205 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.125:46537: Network error: Client connection negotiation failed: client connection to 127.10.114.125:46537: connect: Connection refused (error 111)
W20260925 09:29:41.121383 24212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:41.121627 24212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:41.121708 24212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:41.128408 24212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:29:41.128496 24212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:41.128552 24212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:41.128597 24212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:29:41.128636 24212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:29:41.137513 24212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:42577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:42719,127.10.114.125:46537,127.10.114.124:36245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.125.keytab
--principal=kudu/127.10.114.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.125:46537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.114.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:41.138671 24212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:41.139902 24212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 09:29:41.149314 24205 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } attempt: 4
I20260925 09:29:41.150445 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.149955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:46537 (local address 127.0.0.1:38604)
0925 09:29:41.150238 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.150311 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:46537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260925 09:29:41.151762 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.151181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:46537 (local address 127.0.0.1:38616)
0925 09:29:41.151433 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.151557 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:46537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260925 09:29:41.151144 24218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:41.152020 24205 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.125:46537: Network error: Client connection negotiation failed: client connection to 127.10.114.125:46537: connect: Connection refused (error 111)
W20260925 09:29:41.152127 24217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:41.153064 24220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:41.153443 24212 server_base.cc:1034] running on GCE node
I20260925 09:29:41.154202 24212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:41.155454 24212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:41.156675 24212 hybrid_clock.cc:648] HybridClock initialized: now 1790328581156601 us; error 160 us; skew 500 ppm
I20260925 09:29:41.156994 24212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:41 dist-test-slave-9pj8 krb5kdc[24125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328581, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:41.159951 24212 init.cc:377] Logged in from keytab as kudu/127.10.114.125@KRBTEST.COM (short username kudu)
I20260925 09:29:41.161252 24212 webserver.cc:460] Webserver started at http://127.10.114.125:37605/ using document root <none> and password file <none>
I20260925 09:29:41.161897 24212 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:41.162034 24212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:29:41.162312 24212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:29:41.165122 24212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-1/data/instance:
uuid: "b4f5ee8922e64daa9738971afccf94a1"
format_stamp: "Formatted at 2026-09-25 09:29:41 on dist-test-slave-9pj8"
server_key: "5f34597b9fbea8c482d3615699968d8f"
server_key_iv: "1acbf06826ecf55f24986ed148f7e9fb"
server_key_version: "encryptionkey@0"
I20260925 09:29:41.165879 24212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-1/wal/instance:
uuid: "b4f5ee8922e64daa9738971afccf94a1"
format_stamp: "Formatted at 2026-09-25 09:29:41 on dist-test-slave-9pj8"
server_key: "5f34597b9fbea8c482d3615699968d8f"
server_key_iv: "1acbf06826ecf55f24986ed148f7e9fb"
server_key_version: "encryptionkey@0"
I20260925 09:29:41.170650 24212 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260925 09:29:41.173877 24227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:41.175097 24212 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:29:41.175273 24212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-1/wal
uuid: "b4f5ee8922e64daa9738971afccf94a1"
format_stamp: "Formatted at 2026-09-25 09:29:41 on dist-test-slave-9pj8"
server_key: "5f34597b9fbea8c482d3615699968d8f"
server_key_iv: "1acbf06826ecf55f24986ed148f7e9fb"
server_key_version: "encryptionkey@0"
I20260925 09:29:41.175477 24212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:41.187314 24212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:41.191213 24212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:41.191483 24212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:41.209688 24212 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.125:46537
I20260925 09:29:41.209766 24278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.125:46537 every 8 connection(s)
I20260925 09:29:41.210965 24212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-1/data/info.pb
I20260925 09:29:41.215754 24279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:29:41.216773 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 24212
I20260925 09:29:41.216966 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-1/wal/instance
I20260925 09:29:41.217521 10697 external_mini_cluster.cc:1428] Setting key 751e7351b59482eea8f94b7cb3bca7a5
I20260925 09:29:41.223429 24279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.114.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.124@KRBTEST.COM" created.
Sep 25 09:29:41 dist-test-slave-9pj8 krb5kdc[24125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328581, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.125@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:29:41.256435 24209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.226186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42002 (local address 127.10.114.126:42719)
0925 09:29:41.226440 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:41.226448 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:41.227756 (+ 1308us) server_negotiation.cc:407] Connection header received
0925 09:29:41.230783 (+ 3027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:41.230793 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:41.230891 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:41.231016 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:41.233103 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.234261 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.237004 (+ 2743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.237838 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.237973 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.248962 (+ 10989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:41.248971 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:41.248979 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:41.249029 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:41.252503 (+ 3474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.253323 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.253331 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.253336 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.253509 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.254081 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.254088 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.254094 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.254556 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:41.254743 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:41.255638 (+ 895us) server_negotiation.cc:299] Negotiation successful
0925 09:29:41.255946 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:29:41.257541 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.226667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42719 (local address 127.0.0.1:42002)
0925 09:29:41.227573 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.227628 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:41.230454 (+ 2826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:41.231269 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:41.231282 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:41.232087 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:41.232878 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:41.232904 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.234475 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.234484 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:41.236798 (+ 2314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:41.236819 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.238384 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.238395 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:41.238756 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.245385 (+ 6629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:41.245418 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:41.248655 (+ 3237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:41.252658 (+ 4003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:41.252666 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:41.252683 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:41.253166 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:41.253678 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:41.253687 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:41.253707 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:41.253940 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:41.254716 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:41.254727 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:41.255184 (+ 457us) client_negotiation.cc:769] Sending connection context
0925 09:29:41.255480 (+ 296us) client_negotiation.cc:240] Negotiation successful
0925 09:29:41.256097 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":382,"threads_started":1}
I20260925 09:29:41.259680 24279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:29:41 dist-test-slave-9pj8 krb5kdc[24125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328581, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.125@KRBTEST.COM for kudu/127.10.114.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.124.keytab.
Entry for principal kudu/127.10.114.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.124.keytab.
I20260925 09:29:41.275254 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.124:36245
--webserver_interface=127.10.114.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.124.keytab
--principal=kudu/127.10.114.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:42577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:42719,127.10.114.125:46537,127.10.114.124:36245
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:29:41.278378 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.260471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:46537 (local address 127.0.0.1:38628)
0925 09:29:41.260652 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.260671 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:41.260777 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:41.262100 (+ 1323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:41.262108 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:41.262617 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:41.262835 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:41.262850 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.264999 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.265007 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:41.266447 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:41.266481 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.267670 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.267678 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:41.268028 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.268647 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:41.268673 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:41.271361 (+ 2688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:41.274928 (+ 3567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:41.274937 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:41.274944 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:41.275549 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:41.276009 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:41.276016 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:41.276022 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:41.276253 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:41.277005 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:41.277012 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:41.277198 (+ 186us) client_negotiation.cc:769] Sending connection context
0925 09:29:41.277377 (+ 179us) client_negotiation.cc:240] Negotiation successful
0925 09:29:41.277811 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 09:29:41.278810 24284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.260734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38628 (local address 127.10.114.125:46537)
0925 09:29:41.261511 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:41.261521 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:41.261558 (+ 37us) server_negotiation.cc:407] Connection header received
0925 09:29:41.261618 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:41.261637 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:41.261909 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:41.262080 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:41.263024 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.264833 (+ 1809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.266634 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.267501 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.267645 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.271606 (+ 3961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:41.271617 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:41.271629 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:41.271665 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:41.274755 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.275701 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.275712 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.275715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.275888 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.276385 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.276392 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.276399 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.276853 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:41.277051 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:41.277549 (+ 498us) server_negotiation.cc:299] Negotiation successful
0925 09:29:41.277989 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":153,"server-negotiator.queue_time_us":551,"thread_start_us":379,"threads_started":1}
I20260925 09:29:41.281957 24279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:29:41.283288 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.282762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53290)
0925 09:29:41.282963 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.283092 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260925 09:29:41.283576 24231 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
I20260925 09:29:41.285236 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.284838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53300)
0925 09:29:41.285002 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.285107 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260925 09:29:41.285502 24279 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:36245: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
I20260925 09:29:41.334843 24279 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } attempt: 1
I20260925 09:29:41.336205 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.335566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53314)
0925 09:29:41.335839 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.335976 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260925 09:29:41.337484 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.337076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53324)
0925 09:29:41.337258 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.337351 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260925 09:29:41.337745 24279 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:36245: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
I20260925 09:29:41.405989 24279 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } attempt: 2
I20260925 09:29:41.407196 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.406757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53340)
0925 09:29:41.406950 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.407038 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260925 09:29:41.408360 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.407980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53346)
0925 09:29:41.408140 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.408207 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260925 09:29:41.408546 24279 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:36245: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
I20260925 09:29:41.451275 24205 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } attempt: 5
Sep 25 09:29:41 dist-test-slave-9pj8 krb5kdc[24125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328580, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for kudu/127.10.114.125@KRBTEST.COM
I20260925 09:29:41.466259 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.452087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:46537 (local address 127.0.0.1:38636)
0925 09:29:41.452347 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.452366 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:41.452475 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:41.452820 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:41.452824 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:41.453172 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:41.453406 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:41.453415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.454873 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.454878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:41.455756 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:41.455767 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.456822 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.456827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:41.456965 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.457692 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:41.457717 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:41.460134 (+ 2417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:41.463902 (+ 3768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:41.463905 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:41.463908 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:41.464355 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:41.464734 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:41.464737 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:41.464739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:41.464877 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:41.465519 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:41.465522 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:41.465655 (+ 133us) client_negotiation.cc:769] Sending connection context
0925 09:29:41.465812 (+ 157us) client_negotiation.cc:240] Negotiation successful
0925 09:29:41.466005 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260925 09:29:41.466640 24284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.452192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38636 (local address 127.10.114.125:46537)
0925 09:29:41.452430 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:41.452435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:41.452459 (+ 24us) server_negotiation.cc:407] Connection header received
0925 09:29:41.452590 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:41.452595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:41.452670 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:41.452783 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:41.453602 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.454732 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.455950 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.456661 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.456797 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.460331 (+ 3534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:41.460337 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:41.460342 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:41.460382 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:41.463746 (+ 3364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.464472 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.464478 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.464482 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.464617 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.465010 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.465018 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.465023 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.465396 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:41.465537 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:41.465974 (+ 437us) server_negotiation.cc:299] Negotiation successful
0925 09:29:41.466238 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 09:29:41.467861 24205 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:29:41.469038 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.468635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53356)
0925 09:29:41.468820 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.468905 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260925 09:29:41.470140 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.469776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53370)
0925 09:29:41.469949 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.470013 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260925 09:29:41.470351 24205 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:36245: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
I20260925 09:29:41.499734 24279 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } attempt: 3
I20260925 09:29:41.501004 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.500472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53380)
0925 09:29:41.500687 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.500789 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260925 09:29:41.502151 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.501692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53392)
0925 09:29:41.501922 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.502023 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260925 09:29:41.502352 24279 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:36245: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
I20260925 09:29:41.521589 24205 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } attempt: 1
I20260925 09:29:41.522593 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.522121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53394)
0925 09:29:41.522329 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.522440 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260925 09:29:41.523574 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.523247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53402)
0925 09:29:41.523399 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.523472 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260925 09:29:41.523864 24205 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:36245: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
I20260925 09:29:41.592068 24205 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } attempt: 2
I20260925 09:29:41.593209 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.592678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53416)
0925 09:29:41.592899 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.593010 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260925 09:29:41.594291 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.593917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53422)
0925 09:29:41.594072 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.594161 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 09:29:41.594547 24205 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:36245: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
W20260925 09:29:41.602245 24285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:41.602523 24285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:41.602680 24285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:41.609620 24285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:29:41.609704 24285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:41.609774 24285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:41.609862 24285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:29:41.609932 24285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:29:41.618899 24285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:42577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:42719,127.10.114.125:46537,127.10.114.124:36245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.124.keytab
--principal=kudu/127.10.114.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.124:36245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.114.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:41.620113 24285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:41.621280 24285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:41.632299 24292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:41.632480 24294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:41.632341 24291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:41.633706 24285 server_base.cc:1034] running on GCE node
I20260925 09:29:41.634380 24285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:41.635623 24285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:41.636848 24285 hybrid_clock.cc:648] HybridClock initialized: now 1790328581636797 us; error 61 us; skew 500 ppm
I20260925 09:29:41.637163 24285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:41 dist-test-slave-9pj8 krb5kdc[24125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328581, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:41.640164 24285 init.cc:377] Logged in from keytab as kudu/127.10.114.124@KRBTEST.COM (short username kudu)
I20260925 09:29:41.641491 24285 webserver.cc:460] Webserver started at http://127.10.114.124:46495/ using document root <none> and password file <none>
I20260925 09:29:41.642069 24285 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:41.642156 24285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:29:41.642396 24285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:29:41.645179 24285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-2/data/instance:
uuid: "b2c40029ceb44c51ba0519b643136c22"
format_stamp: "Formatted at 2026-09-25 09:29:41 on dist-test-slave-9pj8"
server_key: "ae7d130f5536754aaa5f205d8c7f60fb"
server_key_iv: "cdb2c30f35c413164908a447054d96a4"
server_key_version: "encryptionkey@0"
I20260925 09:29:41.645606 24279 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } attempt: 4
I20260925 09:29:41.646088 24285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-2/wal/instance:
uuid: "b2c40029ceb44c51ba0519b643136c22"
format_stamp: "Formatted at 2026-09-25 09:29:41 on dist-test-slave-9pj8"
server_key: "ae7d130f5536754aaa5f205d8c7f60fb"
server_key_iv: "cdb2c30f35c413164908a447054d96a4"
server_key_version: "encryptionkey@0"
I20260925 09:29:41.646682 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.646189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53426)
0925 09:29:41.646448 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.646529 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260925 09:29:41.647662 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.647366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53430)
0925 09:29:41.647495 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.647561 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260925 09:29:41.648013 24279 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:36245: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
I20260925 09:29:41.651255 24285 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260925 09:29:41.654608 24301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:41.656033 24285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:29:41.656208 24285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-2/wal
uuid: "b2c40029ceb44c51ba0519b643136c22"
format_stamp: "Formatted at 2026-09-25 09:29:41 on dist-test-slave-9pj8"
server_key: "ae7d130f5536754aaa5f205d8c7f60fb"
server_key_iv: "cdb2c30f35c413164908a447054d96a4"
server_key_version: "encryptionkey@0"
I20260925 09:29:41.656395 24285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:41.685205 24285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:41.689172 24285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:41.689453 24285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:41.700783 24205 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } attempt: 3
I20260925 09:29:41.702049 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.701447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53446)
0925 09:29:41.701768 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.701862 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260925 09:29:41.703379 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.702904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53460)
0925 09:29:41.703103 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.703221 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260925 09:29:41.703629 24205 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:36245: Network error: Client connection negotiation failed: client connection to 127.10.114.124:36245: connect: Connection refused (error 111)
I20260925 09:29:41.708662 24285 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.124:36245
I20260925 09:29:41.708736 24352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.124:36245 every 8 connection(s)
I20260925 09:29:41.710027 24285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-2/data/info.pb
I20260925 09:29:41.713557 24353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:29:41.717881 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 24285
I20260925 09:29:41.718075 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790328385296020-10697-0/minicluster-data/master-2/wal/instance
I20260925 09:29:41.718658 10697 external_mini_cluster.cc:1428] Setting key 845739257f1c5f6080750a77a6554ad1
I20260925 09:29:41.720650 24353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:29:41 dist-test-slave-9pj8 krb5kdc[24125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328581, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.124@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
Sep 25 09:29:41 dist-test-slave-9pj8 krb5kdc[24125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.124@KRBTEST.COM
I20260925 09:29:41.738776 24209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.723220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42018 (local address 127.10.114.126:42719)
0925 09:29:41.723408 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:41.723413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:41.724195 (+ 782us) server_negotiation.cc:407] Connection header received
0925 09:29:41.724457 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:41.724465 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:41.724554 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:41.724662 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:41.726378 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.727022 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.728711 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.729126 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.729243 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.733015 (+ 3772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:41.733021 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:41.733026 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:41.733063 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:41.735967 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.736649 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.736654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.736657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.736789 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.737207 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.737212 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.737215 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.737471 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:41.737616 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:41.738256 (+ 640us) server_negotiation.cc:299] Negotiation successful
0925 09:29:41.738460 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:29:41.739892 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.723289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42719 (local address 127.0.0.1:42018)
0925 09:29:41.724047 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.724094 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:41.724302 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:41.724769 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:41.724774 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:41.725451 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:41.726187 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:41.726215 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.727186 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.727193 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:41.728568 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:41.728581 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.729367 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.729376 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:41.729536 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.730428 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:41.730449 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:41.732829 (+ 2380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:41.736094 (+ 3265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:41.736106 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:41.736116 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:41.736532 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:41.736894 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:41.736899 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:41.736903 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:41.737080 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:41.737590 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:41.737597 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:41.737889 (+ 292us) client_negotiation.cc:769] Sending connection context
0925 09:29:41.738126 (+ 237us) client_negotiation.cc:240] Negotiation successful
0925 09:29:41.738524 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":92,"thread_start_us":332,"threads_started":1}
I20260925 09:29:41.739908 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.720576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53462 (local address 127.10.114.124:36245)
0925 09:29:41.721343 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:41.721357 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:41.721399 (+ 42us) server_negotiation.cc:407] Connection header received
0925 09:29:41.723098 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:41.723125 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:41.723594 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:41.724025 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:41.725610 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.727288 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.728748 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.729309 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.729492 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.733566 (+ 4074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:41.733573 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:41.733583 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:41.733610 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:41.736258 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.736994 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.737002 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.737009 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.737156 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.737561 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.737571 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.737573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.737976 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:41.738273 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:41.738603 (+ 330us) server_negotiation.cc:299] Negotiation successful
0925 09:29:41.738852 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":561,"thread_start_us":359,"threads_started":1}
I20260925 09:29:41.741396 24353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:29:41 dist-test-slave-9pj8 krb5kdc[24125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328581, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.124@KRBTEST.COM for kudu/127.10.114.125@KRBTEST.COM
Sep 25 09:29:41 dist-test-slave-9pj8 krb5kdc[24125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:29:41.755927 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.741984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:46537 (local address 127.0.0.1:38640)
0925 09:29:41.742116 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.742134 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:41.742232 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:41.742637 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:41.742642 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:41.742955 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:41.743132 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:41.743144 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.744570 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.744574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:41.745761 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:41.745775 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.746711 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.746715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:41.746846 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.747350 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:41.747369 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:41.749782 (+ 2413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:41.753401 (+ 3619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:41.753407 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:41.753410 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:41.753889 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:41.754324 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:41.754328 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:41.754331 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:41.754505 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:41.755066 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:41.755071 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:41.755204 (+ 133us) client_negotiation.cc:769] Sending connection context
0925 09:29:41.755368 (+ 164us) client_negotiation.cc:240] Negotiation successful
0925 09:29:41.755591 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 09:29:41.756088 24284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.742103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38640 (local address 127.10.114.125:46537)
0925 09:29:41.742302 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:41.742309 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:41.742329 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:29:41.742399 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:41.742404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:41.742477 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:41.742576 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:41.743350 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.744403 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.745908 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.746581 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.746707 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.749985 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:41.749992 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:41.749996 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:41.750035 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:41.753268 (+ 3233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.754021 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.754025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.754026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.754193 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.754636 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.754643 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.754646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.754946 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:41.755085 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:41.755482 (+ 397us) server_negotiation.cc:299] Negotiation successful
0925 09:29:41.755689 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 09:29:41.757169 24353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:29:41.758517 24209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.743150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42024 (local address 127.10.114.126:42719)
0925 09:29:41.743313 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:41.743319 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:41.743334 (+ 15us) server_negotiation.cc:407] Connection header received
0925 09:29:41.743488 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:41.743491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:41.743537 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:41.743604 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:41.744971 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.745958 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.747634 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.748131 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.748275 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.752474 (+ 4199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:41.752488 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:41.752491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:41.752516 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:41.755535 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.756369 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.756374 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.756377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.756524 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.756932 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.756936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.756939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.757262 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:41.757510 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:41.757994 (+ 484us) server_negotiation.cc:299] Negotiation successful
0925 09:29:41.758186 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Sep 25 09:29:41 dist-test-slave-9pj8 krb5kdc[24125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328581, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.124@KRBTEST.COM for kudu/127.10.114.124@KRBTEST.COM
Sep 25 09:29:41 dist-test-slave-9pj8 krb5kdc[24125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.125@KRBTEST.COM
I20260925 09:29:41.771044 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.757742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53472)
0925 09:29:41.757877 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.757894 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:41.757979 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:41.758462 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:41.758467 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:41.758826 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:41.759010 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:41.759021 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.760300 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.760304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:41.761640 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:41.761657 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.762439 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.762443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:41.762573 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.763064 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:41.763078 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:41.765366 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:41.768913 (+ 3547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:41.768916 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:41.768918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:41.769212 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:41.769555 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:41.769559 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:41.769562 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:41.769676 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:41.770128 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:41.770132 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:41.770272 (+ 140us) client_negotiation.cc:769] Sending connection context
0925 09:29:41.770405 (+ 133us) client_negotiation.cc:240] Negotiation successful
0925 09:29:41.770670 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260925 09:29:41.771086 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.758088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53472 (local address 127.10.114.124:36245)
0925 09:29:41.758234 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:41.758237 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:41.758249 (+ 12us) server_negotiation.cc:407] Connection header received
0925 09:29:41.758279 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:41.758281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:41.758322 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:41.758404 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:41.759147 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.760163 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.761804 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.762301 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.762409 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.765577 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:41.765583 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:41.765587 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:41.765619 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:41.768796 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.769318 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.769325 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.769329 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.769433 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.769781 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.769787 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.769790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.770007 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:41.770158 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:41.770499 (+ 341us) server_negotiation.cc:299] Negotiation successful
0925 09:29:41.770804 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":57}
I20260925 09:29:41.775454 24353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22: Bootstrap starting.
I20260925 09:29:41.775509 24284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.760176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38652 (local address 127.10.114.125:46537)
0925 09:29:41.760321 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:41.760325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:41.760450 (+ 125us) server_negotiation.cc:407] Connection header received
0925 09:29:41.760525 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:41.760529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:41.760593 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:41.760663 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:41.762149 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.763140 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.764793 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.765476 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.765577 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.770269 (+ 4692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:41.770275 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:41.770280 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:41.770324 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:41.773050 (+ 2726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.773662 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.773666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.773669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.773797 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.774155 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.774160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.774163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.774436 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:41.774692 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:41.774952 (+ 260us) server_negotiation.cc:299] Negotiation successful
0925 09:29:41.775200 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 09:29:41.779403 24353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22: Neither blocks nor log segments found. Creating new log.
I20260925 09:29:41.780330 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:29:41.780682 24353 log.cc:824] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22: Log is configured to *not* fsync() on all Append() calls
I20260925 09:29:41.783398 24353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22: No bootstrap required, opened a new log
I20260925 09:29:41.796480 24353 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec98e9821deb4c70a9e470dddf611f1b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } } peers { permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } } peers { permanent_uuid: "b2c40029ceb44c51ba0519b643136c22" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } }
I20260925 09:29:41.797080 24353 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:29:41.797258 24353 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c40029ceb44c51ba0519b643136c22, State: Initialized, Role: FOLLOWER
I20260925 09:29:41.798079 24353 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec98e9821deb4c70a9e470dddf611f1b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } } peers { permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } } peers { permanent_uuid: "b2c40029ceb44c51ba0519b643136c22" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } }
I20260925 09:29:41.804150 24209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.785477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42028 (local address 127.10.114.126:42719)
0925 09:29:41.785653 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:41.785660 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:41.786299 (+ 639us) server_negotiation.cc:407] Connection header received
0925 09:29:41.786490 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:41.786496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:41.786570 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:41.786669 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:41.788827 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.790029 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.792122 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.792789 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.792885 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.796808 (+ 3923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:41.796816 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:41.796821 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:41.796862 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:41.800847 (+ 3985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.801709 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.801715 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.801718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.801869 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.802344 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.802350 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.802353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.802701 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:41.802933 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:41.803533 (+ 600us) server_negotiation.cc:299] Negotiation successful
0925 09:29:41.803770 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 09:29:41.805610 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.785506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53488 (local address 127.10.114.124:36245)
0925 09:29:41.785704 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:41.785725 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:41.788299 (+ 2574us) server_negotiation.cc:407] Connection header received
0925 09:29:41.788595 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:41.788600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:41.788670 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:41.788750 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:41.791086 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.792192 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.794334 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.795123 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.795225 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.798734 (+ 3509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:41.798741 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:41.798750 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:41.798801 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:41.802401 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.803204 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.803209 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.803212 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.803400 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.803934 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.803939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.803943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.804322 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:41.804539 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:41.805039 (+ 500us) server_negotiation.cc:299] Negotiation successful
0925 09:29:41.805220 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 09:29:41.808019 24366 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec98e9821deb4c70a9e470dddf611f1b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } } peers { permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } } peers { permanent_uuid: "b2c40029ceb44c51ba0519b643136c22" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } } }
I20260925 09:29:41.808449 24366 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:29:41.809199 24353 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:29:41.812785 24284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.788414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38668 (local address 127.10.114.125:46537)
0925 09:29:41.788591 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:41.788597 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:41.795003 (+ 6406us) server_negotiation.cc:407] Connection header received
0925 09:29:41.795399 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:41.795404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:41.795473 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:41.795539 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:41.797530 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.798556 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.801159 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.802039 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.802153 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.806398 (+ 4245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:41.806408 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:41.806416 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:41.806473 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:41.809655 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.810536 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.810544 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.810549 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.810719 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.811140 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.811147 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.811152 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.811624 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:41.811959 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:41.812244 (+ 285us) server_negotiation.cc:299] Negotiation successful
0925 09:29:41.812462 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260925 09:29:41.819677 24377 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:29:41.819846 24377 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:29:41.816812 24285 master_runner.cc:186] Error getting master registration for 127.10.114.126:42719: OK, instance_id { permanent_uuid: "ec98e9821deb4c70a9e470dddf611f1b" instance_seqno: 1790328580742691 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 09:29:41.826306 24285 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 09:29:41.880872 24205 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } attempt: 4
Sep 25 09:29:41 dist-test-slave-9pj8 krb5kdc[24125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328580, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for kudu/127.10.114.124@KRBTEST.COM
I20260925 09:29:41.895707 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.881441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53498)
0925 09:29:41.881657 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.881674 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:41.881814 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:41.882188 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:41.882191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:41.882570 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:41.882757 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:41.882765 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.884242 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.884246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:41.885178 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:41.885189 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.886187 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.886191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:41.886328 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.886952 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:41.886970 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:41.889606 (+ 2636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:41.893466 (+ 3860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:41.893470 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:41.893473 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:41.893818 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:41.894212 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:41.894216 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:41.894218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:41.894351 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:41.894983 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:41.894986 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:41.895113 (+ 127us) client_negotiation.cc:769] Sending connection context
0925 09:29:41.895258 (+ 145us) client_negotiation.cc:240] Negotiation successful
0925 09:29:41.895446 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260925 09:29:41.895987 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.881547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53498 (local address 127.10.114.124:36245)
0925 09:29:41.881735 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:41.881742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:41.881762 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:29:41.881933 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:41.881939 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:41.882034 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:41.882132 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:41.882887 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.884105 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.885334 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.886052 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.886173 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.889775 (+ 3602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:41.889781 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:41.889786 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:41.889822 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:41.893332 (+ 3510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.893926 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.893932 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.893936 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.894090 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.894452 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.894458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.894461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.894860 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:41.895001 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:41.895384 (+ 383us) server_negotiation.cc:299] Negotiation successful
0925 09:29:41.895615 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 09:29:41.900488 24205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b: Bootstrap starting.
I20260925 09:29:41.904196 24205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b: Neither blocks nor log segments found. Creating new log.
I20260925 09:29:41.905169 24205 log.cc:824] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b: Log is configured to *not* fsync() on all Append() calls
I20260925 09:29:41.907684 24205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b: No bootstrap required, opened a new log
I20260925 09:29:41.914883 24205 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec98e9821deb4c70a9e470dddf611f1b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } } peers { permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } } peers { permanent_uuid: "b2c40029ceb44c51ba0519b643136c22" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } }
I20260925 09:29:41.915195 24205 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:29:41.915329 24205 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec98e9821deb4c70a9e470dddf611f1b, State: Initialized, Role: FOLLOWER
I20260925 09:29:41.915884 24205 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec98e9821deb4c70a9e470dddf611f1b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } } peers { permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } } peers { permanent_uuid: "b2c40029ceb44c51ba0519b643136c22" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } }
I20260925 09:29:41.916839 24380 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec98e9821deb4c70a9e470dddf611f1b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } } peers { permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } } peers { permanent_uuid: "b2c40029ceb44c51ba0519b643136c22" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } } }
I20260925 09:29:41.917212 24380 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:29:41.917961 24205 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [sys.catalog]: configured and running, proceeding with master startup.
W20260925 09:29:41.927443 24391 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:29:41.927561 24391 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:29:41.926759 24138 master_runner.cc:186] Error getting master registration for 127.10.114.125:46537: OK, instance_id { permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" instance_seqno: 1790328581204417 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 09:29:41.938933 24138 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260925 09:29:41.941363 24307 tablet.cc:2369] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:29:41.947276 24279 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } attempt: 5
Sep 25 09:29:41 dist-test-slave-9pj8 krb5kdc[24125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328581, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.125@KRBTEST.COM for kudu/127.10.114.124@KRBTEST.COM
I20260925 09:29:41.964763 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.947853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:36245 (local address 127.0.0.1:53512)
0925 09:29:41.948070 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:41.948103 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:41.948198 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:41.948574 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:41.948579 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:41.949076 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:41.949287 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:41.949298 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.950744 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.950750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:41.952195 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:41.952219 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.954374 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.954381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:41.954584 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.955267 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:41.955288 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:41.957910 (+ 2622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:41.962035 (+ 4125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:41.962041 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:41.962046 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:41.962549 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:41.962988 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:41.962991 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:41.962994 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:41.963147 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:41.963762 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:41.963767 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:41.963949 (+ 182us) client_negotiation.cc:769] Sending connection context
0925 09:29:41.964092 (+ 143us) client_negotiation.cc:240] Negotiation successful
0925 09:29:41.964381 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260925 09:29:41.965098 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:41.947961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53512 (local address 127.10.114.124:36245)
0925 09:29:41.948151 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:41.948157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:41.948176 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:29:41.948332 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:41.948337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:41.948427 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:41.948537 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:41.949432 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.950577 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.953563 (+ 2986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:41.954226 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:41.954340 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:41.958113 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:41.958119 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:41.958124 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:41.958160 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:41.961887 (+ 3727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.962680 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.962685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.962689 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.962846 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:41.963267 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:41.963271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:41.963275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:41.963634 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:41.963813 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:41.964244 (+ 431us) server_negotiation.cc:299] Negotiation successful
0925 09:29:41.964493 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 09:29:41.970679 24279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1: Bootstrap starting.
I20260925 09:29:41.974704 24279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1: Neither blocks nor log segments found. Creating new log.
I20260925 09:29:41.976030 24279 log.cc:824] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1: Log is configured to *not* fsync() on all Append() calls
I20260925 09:29:41.979265 24279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1: No bootstrap required, opened a new log
W20260925 09:29:41.982502 24159 tablet.cc:2369] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:29:41.990041 24279 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec98e9821deb4c70a9e470dddf611f1b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } } peers { permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } } peers { permanent_uuid: "b2c40029ceb44c51ba0519b643136c22" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } }
I20260925 09:29:41.990383 24279 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:29:41.990482 24279 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f5ee8922e64daa9738971afccf94a1, State: Initialized, Role: FOLLOWER
I20260925 09:29:41.991078 24279 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec98e9821deb4c70a9e470dddf611f1b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } } peers { permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } } peers { permanent_uuid: "b2c40029ceb44c51ba0519b643136c22" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } }
I20260925 09:29:41.995257 24394 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec98e9821deb4c70a9e470dddf611f1b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } } peers { permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } } peers { permanent_uuid: "b2c40029ceb44c51ba0519b643136c22" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } } }
I20260925 09:29:41.995674 24394 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:29:41.996487 24279 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:29:42.011868 24212 master_runner.cc:263] No leader master found from master b4f5ee8922e64daa9738971afccf94a1
I20260925 09:29:42.012013 24212 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 09:29:42.021839 24380 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:29:42.022148 24380 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec98e9821deb4c70a9e470dddf611f1b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } } peers { permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } } peers { permanent_uuid: "b2c40029ceb44c51ba0519b643136c22" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } }
W20260925 09:29:42.022616 24405 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:29:42.022774 24405 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:29:42.023676 24380 leader_election.cc:288] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f5ee8922e64daa9738971afccf94a1 (127.10.114.125:46537), b2c40029ceb44c51ba0519b643136c22 (127.10.114.124:36245)
I20260925 09:29:42.024909 24254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ec98e9821deb4c70a9e470dddf611f1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f5ee8922e64daa9738971afccf94a1" is_pre_election: true
I20260925 09:29:42.024905 24328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ec98e9821deb4c70a9e470dddf611f1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c40029ceb44c51ba0519b643136c22" is_pre_election: true
I20260925 09:29:42.025382 24254 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec98e9821deb4c70a9e470dddf611f1b in term 0.
I20260925 09:29:42.025446 24328 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec98e9821deb4c70a9e470dddf611f1b in term 0.
I20260925 09:29:42.025918 24154 leader_election.cc:302] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f5ee8922e64daa9738971afccf94a1, ec98e9821deb4c70a9e470dddf611f1b; no voters:
I20260925 09:29:42.026486 24380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:29:42.026639 24380 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:29:42.026758 24380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:29:42.030004 24380 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec98e9821deb4c70a9e470dddf611f1b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } } peers { permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } } peers { permanent_uuid: "b2c40029ceb44c51ba0519b643136c22" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } }
I20260925 09:29:42.030998 24254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ec98e9821deb4c70a9e470dddf611f1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f5ee8922e64daa9738971afccf94a1"
I20260925 09:29:42.031181 24254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:29:42.034257 24254 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec98e9821deb4c70a9e470dddf611f1b in term 1.
I20260925 09:29:42.034660 24154 leader_election.cc:302] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f5ee8922e64daa9738971afccf94a1, ec98e9821deb4c70a9e470dddf611f1b; no voters:
I20260925 09:29:42.037390 24328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ec98e9821deb4c70a9e470dddf611f1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c40029ceb44c51ba0519b643136c22"
I20260925 09:29:42.037631 24328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:29:42.040866 24328 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec98e9821deb4c70a9e470dddf611f1b in term 1.
I20260925 09:29:42.045169 24380 leader_election.cc:288] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [CANDIDATE]: Term 1 election: Requested vote from peers b4f5ee8922e64daa9738971afccf94a1 (127.10.114.125:46537), b2c40029ceb44c51ba0519b643136c22 (127.10.114.124:36245)
I20260925 09:29:42.045379 24380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:29:42.045516 24380 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [term 1 LEADER]: Becoming Leader. State: Replica: ec98e9821deb4c70a9e470dddf611f1b, State: Running, Role: LEADER
I20260925 09:29:42.046082 24380 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec98e9821deb4c70a9e470dddf611f1b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } } peers { permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } } peers { permanent_uuid: "b2c40029ceb44c51ba0519b643136c22" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } }
I20260925 09:29:42.063254 24407 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec98e9821deb4c70a9e470dddf611f1b. Latest consensus state: current_term: 1 leader_uuid: "ec98e9821deb4c70a9e470dddf611f1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec98e9821deb4c70a9e470dddf611f1b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } } peers { permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } } peers { permanent_uuid: "b2c40029ceb44c51ba0519b643136c22" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } } }
I20260925 09:29:42.063627 24407 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [sys.catalog]: This master's current role is: LEADER
I20260925 09:29:42.073952 24409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:29:42.076086 24409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:29:42.091413 24328 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22 [term 1 FOLLOWER]: Refusing update from remote peer ec98e9821deb4c70a9e470dddf611f1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:29:42.095808 24407 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2c40029ceb44c51ba0519b643136c22" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:29:42.103084 24254 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1 [term 1 FOLLOWER]: Refusing update from remote peer ec98e9821deb4c70a9e470dddf611f1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:29:42.103843 24407 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:29:42.138257 24394 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec98e9821deb4c70a9e470dddf611f1b. Latest consensus state: current_term: 1 leader_uuid: "ec98e9821deb4c70a9e470dddf611f1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec98e9821deb4c70a9e470dddf611f1b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } } peers { permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } } peers { permanent_uuid: "b2c40029ceb44c51ba0519b643136c22" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } } }
I20260925 09:29:42.138577 24394 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:29:42.140246 24410 mvcc.cc:205] Tried to move back new op lower bound from 7333185872234696704 to 7333185872074149888. Current Snapshot: MvccSnapshot[applied={T|T < 7333185872234696704}]
I20260925 09:29:42.145418 24412 mvcc.cc:205] Tried to move back new op lower bound from 7333185872234696704 to 7333185872074149888. Current Snapshot: MvccSnapshot[applied={T|T < 7333185872234696704}]
I20260925 09:29:42.151923 24380 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec98e9821deb4c70a9e470dddf611f1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec98e9821deb4c70a9e470dddf611f1b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } } peers { permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } } peers { permanent_uuid: "b2c40029ceb44c51ba0519b643136c22" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } } }
I20260925 09:29:42.152220 24380 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [sys.catalog]: This master's current role is: LEADER
I20260925 09:29:42.155248 24409 catalog_manager.cc:1347] Generated new cluster ID: 803746bf8b9f47db8a87bb04e7db81c0
I20260925 09:29:42.155345 24409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:29:42.157109 24394 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec98e9821deb4c70a9e470dddf611f1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec98e9821deb4c70a9e470dddf611f1b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } } peers { permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } } peers { permanent_uuid: "b2c40029ceb44c51ba0519b643136c22" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } } }
I20260925 09:29:42.157341 24394 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:29:42.172170 24366 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec98e9821deb4c70a9e470dddf611f1b. Latest consensus state: current_term: 1 leader_uuid: "ec98e9821deb4c70a9e470dddf611f1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec98e9821deb4c70a9e470dddf611f1b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } } peers { permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } } peers { permanent_uuid: "b2c40029ceb44c51ba0519b643136c22" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } } }
I20260925 09:29:42.172464 24366 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:29:42.173659 24407 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec98e9821deb4c70a9e470dddf611f1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec98e9821deb4c70a9e470dddf611f1b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } } peers { permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } } peers { permanent_uuid: "b2c40029ceb44c51ba0519b643136c22" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } } }
I20260925 09:29:42.173943 24407 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b [sys.catalog]: This master's current role is: LEADER
I20260925 09:29:42.176375 24366 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec98e9821deb4c70a9e470dddf611f1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec98e9821deb4c70a9e470dddf611f1b" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42719 } } peers { permanent_uuid: "b4f5ee8922e64daa9738971afccf94a1" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 46537 } } peers { permanent_uuid: "b2c40029ceb44c51ba0519b643136c22" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 36245 } } }
I20260925 09:29:42.176622 24366 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:29:42.199476 24409 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:29:42.201478 24409 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:29:42.274777 24409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec98e9821deb4c70a9e470dddf611f1b: Generated new TSK 0
I20260925 09:29:42.276000 24409 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:29:42.821839 24377 catalog_manager.cc:1259] Loaded cluster ID: 803746bf8b9f47db8a87bb04e7db81c0
I20260925 09:29:42.821938 24377 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22: loading cluster ID for follower catalog manager: success
I20260925 09:29:42.824344 24377 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22: acquiring CA information for follower catalog manager: success
I20260925 09:29:42.825979 24377 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b2c40029ceb44c51ba0519b643136c22: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 09:29:43.024783 24405 catalog_manager.cc:1259] Loaded cluster ID: 803746bf8b9f47db8a87bb04e7db81c0
I20260925 09:29:43.024886 24405 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1: loading cluster ID for follower catalog manager: success
I20260925 09:29:43.027252 24405 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1: acquiring CA information for follower catalog manager: success
I20260925 09:29:43.029012 24405 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b4f5ee8922e64daa9738971afccf94a1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 09:29:43.356520 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 24138
I20260925 09:29:43.357849 24200 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:29:43.472368 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 24138
I20260925 09:29:43.493172 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 24212
I20260925 09:29:43.494284 24274 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:29:43.596935 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 24212
I20260925 09:29:43.616984 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 24285
I20260925 09:29:43.618002 24348 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:29:43.722117 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 24285
2026-09-25T09:29:43Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4689 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:29:43 dist-test-slave-9pj8 krb5kdc[24426](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:29:43 dist-test-slave-9pj8 krb5kdc[24426](info): set up 2 sockets
Sep 25 09:29:43 dist-test-slave-9pj8 krb5kdc[24426](info): commencing operation
krb5kdc: starting...
W20260925 09:29:44.800506 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:29:44 dist-test-slave-9pj8 krb5kdc[24426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:29:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:29:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:29:45.007563 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:34241
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:43979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:34241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:29:45.319670 24439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:45.319939 24439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:45.320109 24439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:45.326771 24439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:29:45.326856 24439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:45.326951 24439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:45.327020 24439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:29:45.327076 24439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:29:45.335952 24439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:43979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:34241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:34241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:45.337154 24439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:45.338402 24439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:45.349779 24444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:45.349707 24447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:45.349830 24445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:45.350612 24439 server_base.cc:1034] running on GCE node
I20260925 09:29:45.351291 24439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:45.352639 24439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:45.353863 24439 hybrid_clock.cc:648] HybridClock initialized: now 1790328585353826 us; error 48 us; skew 500 ppm
I20260925 09:29:45.354182 24439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:45 dist-test-slave-9pj8 krb5kdc[24426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328585, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:45.357059 24439 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:29:45.358345 24439 webserver.cc:460] Webserver started at http://127.10.114.126:32945/ using document root <none> and password file <none>
I20260925 09:29:45.358975 24439 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:45.359117 24439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:29:45.359407 24439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:29:45.362233 24439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "4febecf1b57b42b6ac81e98e898394a2"
format_stamp: "Formatted at 2026-09-25 09:29:45 on dist-test-slave-9pj8"
server_key: "cd99c3513e5260aa6ae4a389905e23ef"
server_key_iv: "23137232180c3917345cd85552139fae"
server_key_version: "encryptionkey@0"
I20260925 09:29:45.362959 24439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "4febecf1b57b42b6ac81e98e898394a2"
format_stamp: "Formatted at 2026-09-25 09:29:45 on dist-test-slave-9pj8"
server_key: "cd99c3513e5260aa6ae4a389905e23ef"
server_key_iv: "23137232180c3917345cd85552139fae"
server_key_version: "encryptionkey@0"
I20260925 09:29:45.368674 24439 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260925 09:29:45.371881 24454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:45.373040 24439 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 09:29:45.373206 24439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "4febecf1b57b42b6ac81e98e898394a2"
format_stamp: "Formatted at 2026-09-25 09:29:45 on dist-test-slave-9pj8"
server_key: "cd99c3513e5260aa6ae4a389905e23ef"
server_key_iv: "23137232180c3917345cd85552139fae"
server_key_version: "encryptionkey@0"
I20260925 09:29:45.373405 24439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:45.394367 24439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:45.397924 24439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:45.398180 24439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:45.416607 24439 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:34241
I20260925 09:29:45.416617 24505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:34241 every 8 connection(s)
I20260925 09:29:45.417932 24439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:29:45.421393 24506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:29:45.425806 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 24439
I20260925 09:29:45.426034 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:29:45.426891 10697 external_mini_cluster.cc:1428] Setting key e7b3e97b14784a8040ce89a3ba7409c5
I20260925 09:29:45.432412 24506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2: Bootstrap starting.
I20260925 09:29:45.435881 24506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2: Neither blocks nor log segments found. Creating new log.
I20260925 09:29:45.437070 24506 log.cc:824] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2: Log is configured to *not* fsync() on all Append() calls
I20260925 09:29:45.440357 24506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2: No bootstrap required, opened a new log
Sep 25 09:29:45 dist-test-slave-9pj8 krb5kdc[24426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:29:45.450475 24506 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4febecf1b57b42b6ac81e98e898394a2" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 34241 } }
I20260925 09:29:45.450870 24506 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:29:45.450975 24506 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4febecf1b57b42b6ac81e98e898394a2, State: Initialized, Role: FOLLOWER
I20260925 09:29:45.451592 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:45.428699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41752 (local address 127.10.114.126:34241)
0925 09:29:45.429542 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:45.429557 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:45.429602 (+ 45us) server_negotiation.cc:407] Connection header received
0925 09:29:45.431395 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:45.431416 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:45.431815 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:45.432149 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:45.433927 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:45.435305 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:45.437292 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:45.438404 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:45.438545 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:45.443276 (+ 4731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:45.443286 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:45.443304 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:45.443346 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:45.447394 (+ 4048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:45.448139 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:45.448148 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:45.448156 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:45.448368 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:45.448724 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:45.448729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:45.448732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:45.449376 (+ 644us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:45.449683 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:45.449980 (+ 297us) server_negotiation.cc:299] Negotiation successful
0925 09:29:45.450357 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":409,"threads_started":1}
I20260925 09:29:45.451586 24506 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4febecf1b57b42b6ac81e98e898394a2" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 34241 } }
I20260925 09:29:45.451941 24506 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:29:45.452078 24506 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:29:45.452255 24506 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:29:45.454787 24506 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4febecf1b57b42b6ac81e98e898394a2" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 34241 } }
I20260925 09:29:45.455101 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:29:45.455305 24506 leader_election.cc:302] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4febecf1b57b42b6ac81e98e898394a2; no voters:
I20260925 09:29:45.455910 24506 leader_election.cc:288] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:29:45.456028 24511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:29:45.457643 24506 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:29:45.458721 24511 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2 [term 1 LEADER]: Becoming Leader. State: Replica: 4febecf1b57b42b6ac81e98e898394a2, State: Running, Role: LEADER
I20260925 09:29:45.460150 24511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4febecf1b57b42b6ac81e98e898394a2" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 34241 } }
I20260925 09:29:45.462821 24513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4febecf1b57b42b6ac81e98e898394a2. Latest consensus state: current_term: 1 leader_uuid: "4febecf1b57b42b6ac81e98e898394a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4febecf1b57b42b6ac81e98e898394a2" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 34241 } } }
I20260925 09:29:45.463207 24513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2 [sys.catalog]: This master's current role is: LEADER
I20260925 09:29:45.462831 24512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4febecf1b57b42b6ac81e98e898394a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4febecf1b57b42b6ac81e98e898394a2" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 34241 } } }
I20260925 09:29:45.463568 24512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2 [sys.catalog]: This master's current role is: LEADER
I20260925 09:29:45.464773 24522 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:29:45.468981 24522 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:29:45.476379 24522 catalog_manager.cc:1347] Generated new cluster ID: 65c7998a06c34277a7d2e6068bd041a2
I20260925 09:29:45.476511 24522 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:29:45.495666 24522 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:29:45.496969 24522 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:29:45.502028 24522 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4febecf1b57b42b6ac81e98e898394a2: Generated new TSK 0
I20260925 09:29:45.502776 24522 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:29:45.516080 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 24439
I20260925 09:29:45.517102 24501 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:29:45.607811 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 24439
2026-09-25T09:29:45Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1881 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:29:45 dist-test-slave-9pj8 krb5kdc[24534](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:29:45 dist-test-slave-9pj8 krb5kdc[24534](info): set up 2 sockets
Sep 25 09:29:45 dist-test-slave-9pj8 krb5kdc[24534](info): commencing operation
krb5kdc: starting...
W20260925 09:29:46.736883 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:29:46 dist-test-slave-9pj8 krb5kdc[24534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:29:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:29:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:29:46.949506 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:38571
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:38677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:38571
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:29:47.261504 24547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:47.261780 24547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:47.261869 24547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:47.268716 24547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:29:47.268807 24547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:47.268850 24547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:47.268904 24547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:29:47.268946 24547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:29:47.278370 24547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:38571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:38571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:47.279569 24547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:47.280805 24547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:47.292258 24552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:47.292150 24553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:47.292295 24555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:47.292999 24547 server_base.cc:1034] running on GCE node
I20260925 09:29:47.293800 24547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:47.295173 24547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:47.296415 24547 hybrid_clock.cc:648] HybridClock initialized: now 1790328587296378 us; error 49 us; skew 500 ppm
I20260925 09:29:47.296831 24547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:47 dist-test-slave-9pj8 krb5kdc[24534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328587, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:47.303572 24547 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:29:47.305197 24547 webserver.cc:460] Webserver started at http://127.10.114.126:37863/ using document root <none> and password file <none>
I20260925 09:29:47.305919 24547 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:47.306077 24547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:29:47.306401 24547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:29:47.309720 24547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "d81c17443e4c4671b82396246ea49169"
format_stamp: "Formatted at 2026-09-25 09:29:47 on dist-test-slave-9pj8"
server_key: "5169b08cfd8670a31beccd733fb375fc"
server_key_iv: "34b908e1780aa450eefd3c4afd2bb2ed"
server_key_version: "encryptionkey@0"
I20260925 09:29:47.310528 24547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "d81c17443e4c4671b82396246ea49169"
format_stamp: "Formatted at 2026-09-25 09:29:47 on dist-test-slave-9pj8"
server_key: "5169b08cfd8670a31beccd733fb375fc"
server_key_iv: "34b908e1780aa450eefd3c4afd2bb2ed"
server_key_version: "encryptionkey@0"
I20260925 09:29:47.315892 24547 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260925 09:29:47.319339 24562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:47.320609 24547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:29:47.320808 24547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "d81c17443e4c4671b82396246ea49169"
format_stamp: "Formatted at 2026-09-25 09:29:47 on dist-test-slave-9pj8"
server_key: "5169b08cfd8670a31beccd733fb375fc"
server_key_iv: "34b908e1780aa450eefd3c4afd2bb2ed"
server_key_version: "encryptionkey@0"
I20260925 09:29:47.321018 24547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:47.341148 24547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:47.345520 24547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:47.345813 24547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:47.363731 24547 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:38571
I20260925 09:29:47.363785 24613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:38571 every 8 connection(s)
I20260925 09:29:47.365061 24547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:29:47.368444 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 24547
I20260925 09:29:47.368528 24614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:29:47.368664 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:29:47.369303 10697 external_mini_cluster.cc:1428] Setting key 7b439aa6d7ac5a8931c6e75915995fd6
I20260925 09:29:47.382910 24614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169: Bootstrap starting.
Sep 25 09:29:47 dist-test-slave-9pj8 krb5kdc[24534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:29:47.386672 24614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169: Neither blocks nor log segments found. Creating new log.
I20260925 09:29:47.387706 24614 log.cc:824] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169: Log is configured to *not* fsync() on all Append() calls
I20260925 09:29:47.390409 24614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169: No bootstrap required, opened a new log
I20260925 09:29:47.394040 24617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:47.371390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48940 (local address 127.10.114.126:38571)
0925 09:29:47.372257 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:47.372272 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:47.372316 (+ 44us) server_negotiation.cc:407] Connection header received
0925 09:29:47.375278 (+ 2962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:47.375300 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:47.375713 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:47.376132 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:47.377822 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:47.379151 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:47.381235 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:47.381906 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:47.382038 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:47.386794 (+ 4756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:47.386805 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:47.386823 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:47.386870 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:47.389961 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:47.390762 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:47.390769 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:47.390774 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:47.390950 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:47.391335 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:47.391344 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:47.391353 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:47.391934 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:47.392336 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:47.392677 (+ 341us) server_negotiation.cc:299] Negotiation successful
0925 09:29:47.393099 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":381,"threads_started":1}
I20260925 09:29:47.398346 24614 raft_consensus.cc:348] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81c17443e4c4671b82396246ea49169" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 38571 } }
I20260925 09:29:47.398705 24614 raft_consensus.cc:374] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:29:47.398813 24614 raft_consensus.cc:729] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d81c17443e4c4671b82396246ea49169, State: Initialized, Role: FOLLOWER
I20260925 09:29:47.399366 24614 consensus_queue.cc:260] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81c17443e4c4671b82396246ea49169" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 38571 } }
I20260925 09:29:47.399595 24614 raft_consensus.cc:388] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:29:47.399735 24614 raft_consensus.cc:482] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:29:47.399914 24614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:29:47.401916 24614 raft_consensus.cc:504] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81c17443e4c4671b82396246ea49169" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 38571 } }
I20260925 09:29:47.402380 24614 leader_election.cc:302] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d81c17443e4c4671b82396246ea49169; no voters:
I20260925 09:29:47.402961 24614 leader_election.cc:288] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:29:47.403120 24619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:29:47.403800 24619 raft_consensus.cc:686] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 1 LEADER]: Becoming Leader. State: Replica: d81c17443e4c4671b82396246ea49169, State: Running, Role: LEADER
I20260925 09:29:47.404361 24619 consensus_queue.cc:237] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81c17443e4c4671b82396246ea49169" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 38571 } }
I20260925 09:29:47.404915 24614 sys_catalog.cc:564] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:29:47.407059 24620 sys_catalog.cc:455] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d81c17443e4c4671b82396246ea49169" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81c17443e4c4671b82396246ea49169" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 38571 } } }
I20260925 09:29:47.407410 24620 sys_catalog.cc:458] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [sys.catalog]: This master's current role is: LEADER
I20260925 09:29:47.408960 24625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:29:47.409062 24621 sys_catalog.cc:455] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d81c17443e4c4671b82396246ea49169. Latest consensus state: current_term: 1 leader_uuid: "d81c17443e4c4671b82396246ea49169" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81c17443e4c4671b82396246ea49169" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 38571 } } }
I20260925 09:29:47.409363 24621 sys_catalog.cc:458] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [sys.catalog]: This master's current role is: LEADER
I20260925 09:29:47.413349 24625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:29:47.422811 24625 catalog_manager.cc:1347] Generated new cluster ID: fd8036fd223d41dc910de92db291d028
I20260925 09:29:47.422914 24625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:29:47.436269 24625 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:29:47.437516 24625 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:29:47.445495 24625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169: Generated new TSK 0
I20260925 09:29:47.446256 24625 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
I20260925 09:29:47.507349 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:38571
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:38677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:29:47.837051 24640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:47.837352 24640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:47.837515 24640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:47.844559 24640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:47.844702 24640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:47.844874 24640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:29:47.854118 24640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:38571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:47.855273 24640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:47.856560 24640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:47.868894 24645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:47.868868 24648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:47.868923 24646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:47.870272 24640 server_base.cc:1034] running on GCE node
I20260925 09:29:47.870824 24640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:47.871842 24640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:47.873073 24640 hybrid_clock.cc:648] HybridClock initialized: now 1790328587873023 us; error 61 us; skew 500 ppm
I20260925 09:29:47.873377 24640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:47 dist-test-slave-9pj8 krb5kdc[24534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328587, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:47.876529 24640 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:29:47.877830 24640 webserver.cc:460] Webserver started at http://127.10.114.65:34325/ using document root <none> and password file <none>
I20260925 09:29:47.878417 24640 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:47.878510 24640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:29:47.878727 24640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:29:47.881536 24640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "8df6aef7bde44bf39163ca13c1810d6b"
format_stamp: "Formatted at 2026-09-25 09:29:47 on dist-test-slave-9pj8"
server_key: "2417cd28b83f1c8ad7ad507ddd28e94d"
server_key_iv: "4b3c54f0dd36291ec0c1a5ce5e96b79a"
server_key_version: "encryptionkey@0"
I20260925 09:29:47.882239 24640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "8df6aef7bde44bf39163ca13c1810d6b"
format_stamp: "Formatted at 2026-09-25 09:29:47 on dist-test-slave-9pj8"
server_key: "2417cd28b83f1c8ad7ad507ddd28e94d"
server_key_iv: "4b3c54f0dd36291ec0c1a5ce5e96b79a"
server_key_version: "encryptionkey@0"
I20260925 09:29:47.886899 24640 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260925 09:29:47.890094 24655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:47.891402 24640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:29:47.891561 24640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "8df6aef7bde44bf39163ca13c1810d6b"
format_stamp: "Formatted at 2026-09-25 09:29:47 on dist-test-slave-9pj8"
server_key: "2417cd28b83f1c8ad7ad507ddd28e94d"
server_key_iv: "4b3c54f0dd36291ec0c1a5ce5e96b79a"
server_key_version: "encryptionkey@0"
I20260925 09:29:47.891743 24640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:47.912007 24640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:47.916364 24640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:47.916705 24640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:47.917625 24640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:29:47.918962 24640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:29:47.919065 24640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:47.919250 24640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:29:47.919317 24640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:47.953388 24640 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:46239
I20260925 09:29:47.953425 24767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:46239 every 8 connection(s)
I20260925 09:29:47.954711 24640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:29:47.960807 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 24640
I20260925 09:29:47.961004 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:29:47.961812 10697 external_mini_cluster.cc:1428] Setting key 0e3de702921536a0fd877a57f702c367
Sep 25 09:29:47 dist-test-slave-9pj8 krb5kdc[24534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328587, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:29:47.975986 24770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:47.957300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:41871 (local address 127.10.114.126:38571)
0925 09:29:47.957731 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:47.957735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:47.958642 (+ 907us) server_negotiation.cc:407] Connection header received
0925 09:29:47.960531 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:47.960536 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:47.960632 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:47.960727 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:47.962769 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:47.963547 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:47.965939 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:47.966348 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:47.966442 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:47.970408 (+ 3966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:47.970413 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:47.970416 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:47.970446 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:47.973125 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:47.973824 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:47.973835 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:47.973837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:47.973944 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:47.974346 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:47.974350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:47.974353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:47.974618 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:47.974768 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:47.975395 (+ 627us) server_negotiation.cc:299] Negotiation successful
0925 09:29:47.975598 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":232,"threads_started":1}
I20260925 09:29:47.976876 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:47.957755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.65:41871)
0925 09:29:47.958482 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:47.958519 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:47.960196 (+ 1677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:47.961016 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:47.961029 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:47.961771 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:47.962550 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:47.962571 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:47.963704 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:47.963707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:47.965790 (+ 2083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:47.965809 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:47.966573 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:47.966582 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:47.966903 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:47.967616 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:47.967654 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:47.970186 (+ 2532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:47.973265 (+ 3079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:47.973274 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:47.973289 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:47.973705 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:47.974049 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:47.974054 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:47.974058 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:47.974235 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:47.974745 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:47.974752 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:47.975053 (+ 301us) client_negotiation.cc:769] Sending connection context
0925 09:29:47.975291 (+ 238us) client_negotiation.cc:240] Negotiation successful
0925 09:29:47.975796 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":263,"threads_started":1}
I20260925 09:29:47.978238 24768 heartbeater.cc:347] Connected to a master server at 127.10.114.126:38571
I20260925 09:29:47.978523 24768 heartbeater.cc:464] Registering TS with master...
I20260925 09:29:47.979429 24768 heartbeater.cc:511] Master 127.10.114.126:38571 requested a full tablet report, sending...
I20260925 09:29:47.981709 24579 ts_manager.cc:194] Registered new tserver with Master: 8df6aef7bde44bf39163ca13c1810d6b (127.10.114.65:46239)
I20260925 09:29:47.983135 24579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:41871
WARNING: no policy specified for kudu/127.10.114.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
I20260925 09:29:48.013028 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:38571
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:38677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:29:48.325623 24774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:48.325889 24774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:48.325986 24774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:48.333132 24774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:48.333242 24774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:48.333374 24774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:29:48.342615 24774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:38571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:48.343748 24774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:48.345049 24774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:48.358040 24780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:48.358065 24779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:48.358245 24782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:48.358757 24774 server_base.cc:1034] running on GCE node
I20260925 09:29:48.359352 24774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:48.360255 24774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:48.361485 24774 hybrid_clock.cc:648] HybridClock initialized: now 1790328588361426 us; error 40 us; skew 500 ppm
I20260925 09:29:48.361773 24774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:48 dist-test-slave-9pj8 krb5kdc[24534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328588, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:48.365029 24774 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:29:48.366531 24774 webserver.cc:460] Webserver started at http://127.10.114.66:36151/ using document root <none> and password file <none>
I20260925 09:29:48.367173 24774 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:48.367275 24774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:29:48.367585 24774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:29:48.370365 24774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "3193e95317f24e62a1d32918e04e927f"
format_stamp: "Formatted at 2026-09-25 09:29:48 on dist-test-slave-9pj8"
server_key: "b230bb6e140f6fa69b2bbd2b09fdacf9"
server_key_iv: "4dfee59ff957795e9f68c9bb1024c8e3"
server_key_version: "encryptionkey@0"
I20260925 09:29:48.371090 24774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "3193e95317f24e62a1d32918e04e927f"
format_stamp: "Formatted at 2026-09-25 09:29:48 on dist-test-slave-9pj8"
server_key: "b230bb6e140f6fa69b2bbd2b09fdacf9"
server_key_iv: "4dfee59ff957795e9f68c9bb1024c8e3"
server_key_version: "encryptionkey@0"
I20260925 09:29:48.376416 24774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260925 09:29:48.380344 24789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:48.381559 24774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:29:48.381754 24774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "3193e95317f24e62a1d32918e04e927f"
format_stamp: "Formatted at 2026-09-25 09:29:48 on dist-test-slave-9pj8"
server_key: "b230bb6e140f6fa69b2bbd2b09fdacf9"
server_key_iv: "4dfee59ff957795e9f68c9bb1024c8e3"
server_key_version: "encryptionkey@0"
I20260925 09:29:48.381949 24774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:48.401188 24774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:48.404930 24774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:48.405196 24774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:48.406169 24774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:29:48.407421 24774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:29:48.407529 24774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:48.407680 24774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:29:48.407752 24774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:48.442566 24774 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:38941
I20260925 09:29:48.442581 24901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:38941 every 8 connection(s)
I20260925 09:29:48.444123 24774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:29:48.453840 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 24774
I20260925 09:29:48.454027 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:29:48.454614 10697 external_mini_cluster.cc:1428] Setting key 981a91443e25458cb101970123d786d3
Sep 25 09:29:48 dist-test-slave-9pj8 krb5kdc[24534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328588, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:29:48.464557 24770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:48.446656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:33111 (local address 127.10.114.126:38571)
0925 09:29:48.446821 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:48.446826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:48.447992 (+ 1166us) server_negotiation.cc:407] Connection header received
0925 09:29:48.449741 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:48.449746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:48.449833 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:48.449958 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:48.451372 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:48.452077 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:48.453614 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:48.454064 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:48.454195 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:48.457874 (+ 3679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:48.457883 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:48.457891 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:48.457937 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:48.461561 (+ 3624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:48.462191 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:48.462198 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:48.462204 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:48.462355 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:48.462726 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:48.462734 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:48.462739 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:48.463107 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:48.463252 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:48.463876 (+ 624us) server_negotiation.cc:299] Negotiation successful
0925 09:29:48.464131 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 09:29:48.465253 24904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:48.447145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.66:33111)
0925 09:29:48.447851 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:48.447888 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:48.449482 (+ 1594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:48.450197 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:48.450205 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:48.450663 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:48.451208 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:48.451224 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:48.452266 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:48.452271 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:48.453428 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:48.453439 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:48.454312 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:48.454320 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:48.454596 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:48.455309 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:48.455336 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:48.457644 (+ 2308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:48.461688 (+ 4044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:48.461693 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:48.461704 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:48.462078 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:48.462457 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:48.462460 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:48.462462 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:48.462618 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:48.463219 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:48.463224 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:48.463500 (+ 276us) client_negotiation.cc:769] Sending connection context
0925 09:29:48.463714 (+ 214us) client_negotiation.cc:240] Negotiation successful
0925 09:29:48.464203 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":231,"threads_started":1}
I20260925 09:29:48.466666 24902 heartbeater.cc:347] Connected to a master server at 127.10.114.126:38571
I20260925 09:29:48.466969 24902 heartbeater.cc:464] Registering TS with master...
I20260925 09:29:48.467720 24902 heartbeater.cc:511] Master 127.10.114.126:38571 requested a full tablet report, sending...
I20260925 09:29:48.469195 24579 ts_manager.cc:194] Registered new tserver with Master: 3193e95317f24e62a1d32918e04e927f (127.10.114.66:38941)
I20260925 09:29:48.470356 24579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:33111
WARNING: no policy specified for kudu/127.10.114.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
I20260925 09:29:48.507213 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:38571
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:38677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:29:48.821468 24907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:48.821708 24907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:48.821852 24907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:48.828651 24907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:48.828764 24907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:48.828948 24907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:29:48.838121 24907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:38571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:48.839305 24907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:48.840601 24907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:48.852954 24915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:48.853067 24912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:48.853093 24913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:48.854045 24907 server_base.cc:1034] running on GCE node
I20260925 09:29:48.854569 24907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:48.855513 24907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:48.856703 24907 hybrid_clock.cc:648] HybridClock initialized: now 1790328588856628 us; error 66 us; skew 500 ppm
I20260925 09:29:48.857018 24907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:48 dist-test-slave-9pj8 krb5kdc[24534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328588, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:48.859987 24907 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:29:48.861279 24907 webserver.cc:460] Webserver started at http://127.10.114.67:33821/ using document root <none> and password file <none>
I20260925 09:29:48.861882 24907 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:48.861987 24907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:29:48.862201 24907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:29:48.865060 24907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "ca4dc01101f74ee79542718e10cf7df0"
format_stamp: "Formatted at 2026-09-25 09:29:48 on dist-test-slave-9pj8"
server_key: "31fb53e9f9f6c2b00d98291319c3b7dd"
server_key_iv: "498f8088bf22e7fc17bd20470912cfc8"
server_key_version: "encryptionkey@0"
I20260925 09:29:48.865823 24907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "ca4dc01101f74ee79542718e10cf7df0"
format_stamp: "Formatted at 2026-09-25 09:29:48 on dist-test-slave-9pj8"
server_key: "31fb53e9f9f6c2b00d98291319c3b7dd"
server_key_iv: "498f8088bf22e7fc17bd20470912cfc8"
server_key_version: "encryptionkey@0"
I20260925 09:29:48.870661 24907 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:29:48.874014 24922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:48.875486 24907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:29:48.875607 24907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "ca4dc01101f74ee79542718e10cf7df0"
format_stamp: "Formatted at 2026-09-25 09:29:48 on dist-test-slave-9pj8"
server_key: "31fb53e9f9f6c2b00d98291319c3b7dd"
server_key_iv: "498f8088bf22e7fc17bd20470912cfc8"
server_key_version: "encryptionkey@0"
I20260925 09:29:48.875762 24907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:48.908290 24907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:48.912246 24907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:48.912489 24907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:48.913434 24907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:29:48.914695 24907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:29:48.914767 24907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:48.914892 24907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:29:48.914937 24907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:48.948966 24907 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:34895
I20260925 09:29:48.949012 25034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:34895 every 8 connection(s)
I20260925 09:29:48.950255 24907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:29:48.950652 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 24907
I20260925 09:29:48.950848 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:29:48.951445 10697 external_mini_cluster.cc:1428] Setting key 1bd179c3d3dce89a27b2033933e99df7
Sep 25 09:29:48 dist-test-slave-9pj8 krb5kdc[24534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328588, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:29:48.970834 24770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:48.952968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:42157 (local address 127.10.114.126:38571)
0925 09:29:48.953161 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:48.953167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:48.955010 (+ 1843us) server_negotiation.cc:407] Connection header received
0925 09:29:48.956905 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:48.956910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:48.957043 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:48.957147 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:48.958533 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:48.959287 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:48.960884 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:48.961292 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:48.961421 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:48.964710 (+ 3289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:48.964716 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:48.964721 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:48.964761 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:48.967940 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:48.968603 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:48.968608 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:48.968612 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:48.968755 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:48.969154 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:48.969158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:48.969162 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:48.969423 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:48.969606 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:48.970261 (+ 655us) server_negotiation.cc:299] Negotiation successful
0925 09:29:48.970494 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 09:29:48.971655 25037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:48.953701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.67:42157)
0925 09:29:48.954766 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:48.954815 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:48.956646 (+ 1831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:48.957376 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:48.957384 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:48.957865 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:48.958378 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:48.958394 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:48.959437 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:48.959449 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:48.960737 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:48.960748 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:48.961509 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:48.961518 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:48.961741 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:48.962354 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:48.962377 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:48.964512 (+ 2135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:48.968059 (+ 3547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:48.968067 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:48.968081 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:48.968460 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:48.968860 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:48.968865 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:48.968868 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:48.969039 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:48.969562 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:48.969569 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:48.969888 (+ 319us) client_negotiation.cc:769] Sending connection context
0925 09:29:48.970150 (+ 262us) client_negotiation.cc:240] Negotiation successful
0925 09:29:48.970650 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"thread_start_us":590,"threads_started":1}
I20260925 09:29:48.972898 25035 heartbeater.cc:347] Connected to a master server at 127.10.114.126:38571
I20260925 09:29:48.973150 25035 heartbeater.cc:464] Registering TS with master...
I20260925 09:29:48.973856 25035 heartbeater.cc:511] Master 127.10.114.126:38571 requested a full tablet report, sending...
I20260925 09:29:48.975257 24579 ts_manager.cc:194] Registered new tserver with Master: ca4dc01101f74ee79542718e10cf7df0 (127.10.114.67:34895)
I20260925 09:29:48.976164 24579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:42157
I20260925 09:29:48.979735 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:29:48.986491 24768 heartbeater.cc:503] Master 127.10.114.126:38571 was elected leader, sending a full tablet report...
I20260925 09:29:48.997960 24770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:48.984629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48944 (local address 127.10.114.126:38571)
0925 09:29:48.984858 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:48.984867 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:48.985247 (+ 380us) server_negotiation.cc:407] Connection header received
0925 09:29:48.985367 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:48.985370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:48.985425 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:48.985499 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:48.987488 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:48.988461 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:48.989910 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:48.990358 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:48.990465 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:48.992940 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:48.992944 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:48.992948 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:48.992975 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:48.995301 (+ 2326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:48.996034 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:48.996038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:48.996040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:48.996162 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:48.996549 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:48.996553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:48.996555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:48.996845 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:48.997056 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:48.997401 (+ 345us) server_negotiation.cc:299] Negotiation successful
0925 09:29:48.997558 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 09:29:49.001869 24579 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:29:49.009603 24579 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:29:49.032737 25048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.022070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:38941 (local address 127.0.0.1:60632)
0925 09:29:49.023273 (+ 1203us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:49.023319 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:49.023488 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:49.025012 (+ 1524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:49.025024 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:49.025059 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:29:49.025505 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:49.025522 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.027660 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.027665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:49.030095 (+ 2430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:49.030112 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.031660 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.031667 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:49.031934 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:49.031997 (+ 63us) client_negotiation.cc:769] Sending connection context
0925 09:29:49.032102 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 09:29:49.032283 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":935,"mutex_wait_us":3,"thread_start_us":3498,"threads_started":1}
I20260925 09:29:49.033222 25047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.022484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:34895 (local address 127.0.0.1:40304)
0925 09:29:49.025875 (+ 3391us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:49.025895 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:49.026009 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:49.026699 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:49.026710 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:49.026731 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:29:49.027058 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:49.027067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.028908 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.028917 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:49.030674 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:49.030687 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.032425 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.032431 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:49.032584 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:49.032625 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 09:29:49.032715 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 09:29:49.032835 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3240,"thread_start_us":2198,"threads_started":1}
I20260925 09:29:49.033478 25050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.022591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60632 (local address 127.10.114.66:38941)
0925 09:29:49.024247 (+ 1656us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:49.024258 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:49.024289 (+ 31us) server_negotiation.cc:407] Connection header received
0925 09:29:49.024408 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:49.024416 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:49.024730 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:49.025741 (+ 1011us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:29:49.025978 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.027496 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.030252 (+ 2756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.031423 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.032686 (+ 1263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:49.032911 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:49.032972 (+ 61us) server_negotiation.cc:299] Negotiation successful
0925 09:29:49.033135 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1493,"thread_start_us":1380,"threads_started":1}
I20260925 09:29:49.033998 25052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.023844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40304 (local address 127.10.114.67:34895)
0925 09:29:49.026128 (+ 2284us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:49.026139 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:49.026169 (+ 30us) server_negotiation.cc:407] Connection header received
0925 09:29:49.026270 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:49.026278 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:49.026502 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:49.026682 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:29:49.027253 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.028774 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.030843 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.032217 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.032335 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:49.032560 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:49.033484 (+ 924us) server_negotiation.cc:299] Negotiation successful
0925 09:29:49.033636 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2098,"thread_start_us":1949,"threads_started":1}
I20260925 09:29:49.034605 25049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.022143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:46239 (local address 127.0.0.1:33608)
0925 09:29:49.024983 (+ 2840us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:49.025001 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:49.025090 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:49.027446 (+ 2356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:49.027451 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:49.027473 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:29:49.028010 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:49.028026 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.029967 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.029972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:49.032026 (+ 2054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:49.032038 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.033891 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.033896 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:49.034043 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:49.034071 (+ 28us) client_negotiation.cc:769] Sending connection context
0925 09:29:49.034144 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 09:29:49.034263 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2682,"mutex_wait_us":66,"thread_start_us":685,"threads_started":1}
I20260925 09:29:49.035377 25051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.023089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33608 (local address 127.10.114.65:46239)
0925 09:29:49.025367 (+ 2278us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:49.025378 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:49.025405 (+ 27us) server_negotiation.cc:407] Connection header received
0925 09:29:49.025503 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:49.025510 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:49.027033 (+ 1523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:49.027209 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:29:49.028248 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.029820 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.032171 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.033652 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.033833 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:49.034040 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:49.034857 (+ 817us) server_negotiation.cc:299] Negotiation successful
0925 09:29:49.035005 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1409,"thread_start_us":1235,"threads_started":1}
I20260925 09:29:49.037422 24837 tablet_service.cc:1467] Processing CreateTablet for tablet 738ae18afa664f3b94687a2e95225ada (DEFAULT_TABLE table=test-table [id=85645259b5ae48b5806505c2d961b718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:29:49.037422 24970 tablet_service.cc:1467] Processing CreateTablet for tablet 738ae18afa664f3b94687a2e95225ada (DEFAULT_TABLE table=test-table [id=85645259b5ae48b5806505c2d961b718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:29:49.038892 24837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 738ae18afa664f3b94687a2e95225ada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:29:49.038890 24970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 738ae18afa664f3b94687a2e95225ada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:29:49.038535 24703 tablet_service.cc:1467] Processing CreateTablet for tablet 738ae18afa664f3b94687a2e95225ada (DEFAULT_TABLE table=test-table [id=85645259b5ae48b5806505c2d961b718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:29:49.039639 24703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 738ae18afa664f3b94687a2e95225ada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:29:49.050441 25053 tablet_bootstrap.cc:492] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b: Bootstrap starting.
I20260925 09:29:49.052811 25054 tablet_bootstrap.cc:492] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0: Bootstrap starting.
I20260925 09:29:49.053572 25055 tablet_bootstrap.cc:492] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f: Bootstrap starting.
I20260925 09:29:49.054863 25053 tablet_bootstrap.cc:654] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b: Neither blocks nor log segments found. Creating new log.
I20260925 09:29:49.056141 25053 log.cc:824] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b: Log is configured to *not* fsync() on all Append() calls
I20260925 09:29:49.057113 25054 tablet_bootstrap.cc:654] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0: Neither blocks nor log segments found. Creating new log.
I20260925 09:29:49.058308 25054 log.cc:824] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0: Log is configured to *not* fsync() on all Append() calls
I20260925 09:29:49.058630 25055 tablet_bootstrap.cc:654] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f: Neither blocks nor log segments found. Creating new log.
I20260925 09:29:49.059263 25053 tablet_bootstrap.cc:492] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b: No bootstrap required, opened a new log
I20260925 09:29:49.059513 25053 ts_tablet_manager.cc:1397] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260925 09:29:49.059872 25055 log.cc:824] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f: Log is configured to *not* fsync() on all Append() calls
I20260925 09:29:49.061272 25054 tablet_bootstrap.cc:492] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0: No bootstrap required, opened a new log
I20260925 09:29:49.061528 25054 ts_tablet_manager.cc:1397] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260925 09:29:49.062486 25055 tablet_bootstrap.cc:492] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f: No bootstrap required, opened a new log
I20260925 09:29:49.062753 25055 ts_tablet_manager.cc:1397] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260925 09:29:49.069113 25055 raft_consensus.cc:348] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } } peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } }
I20260925 09:29:49.069438 25055 raft_consensus.cc:374] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:29:49.069557 25055 raft_consensus.cc:729] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3193e95317f24e62a1d32918e04e927f, State: Initialized, Role: FOLLOWER
I20260925 09:29:49.069713 25053 raft_consensus.cc:348] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } } peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } }
I20260925 09:29:49.070153 25053 raft_consensus.cc:374] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:29:49.070252 25053 raft_consensus.cc:729] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df6aef7bde44bf39163ca13c1810d6b, State: Initialized, Role: FOLLOWER
I20260925 09:29:49.070273 25055 consensus_queue.cc:260] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } } peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } }
I20260925 09:29:49.070776 25053 consensus_queue.cc:260] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } } peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } }
I20260925 09:29:49.071466 24902 heartbeater.cc:503] Master 127.10.114.126:38571 was elected leader, sending a full tablet report...
I20260925 09:29:49.072064 25055 ts_tablet_manager.cc:1428] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f: Time spent starting tablet: real 0.009s user 0.008s sys 0.001s
I20260925 09:29:49.072989 25053 ts_tablet_manager.cc:1428] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b: Time spent starting tablet: real 0.013s user 0.013s sys 0.002s
I20260925 09:29:49.073536 25054 raft_consensus.cc:348] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } } peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } }
I20260925 09:29:49.073998 25054 raft_consensus.cc:374] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:29:49.074100 25054 raft_consensus.cc:729] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca4dc01101f74ee79542718e10cf7df0, State: Initialized, Role: FOLLOWER
I20260925 09:29:49.074664 25054 consensus_queue.cc:260] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } } peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } }
I20260925 09:29:49.075623 25035 heartbeater.cc:503] Master 127.10.114.126:38571 was elected leader, sending a full tablet report...
I20260925 09:29:49.076264 25054 ts_tablet_manager.cc:1428] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0: Time spent starting tablet: real 0.015s user 0.009s sys 0.005s
W20260925 09:29:49.196142 24903 tablet.cc:2369] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:29:49.201949 25036 tablet.cc:2369] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:29:49.207065 24769 tablet.cc:2369] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:29:49.320849 25060 raft_consensus.cc:482] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:29:49.321166 25060 raft_consensus.cc:504] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } } peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } }
I20260925 09:29:49.322885 25060 leader_election.cc:288] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3193e95317f24e62a1d32918e04e927f (127.10.114.66:38941), ca4dc01101f74ee79542718e10cf7df0 (127.10.114.67:34895)
I20260925 09:29:49.329946 25052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.323448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:34521 (local address 127.10.114.67:34895)
0925 09:29:49.323901 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:49.323909 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:49.324217 (+ 308us) server_negotiation.cc:407] Connection header received
0925 09:29:49.324482 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:49.324486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:49.324552 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:49.324666 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:29:49.325396 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.326158 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.328226 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.328971 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.329130 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:49.329232 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:49.329574 (+ 342us) server_negotiation.cc:299] Negotiation successful
0925 09:29:49.329690 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279}
I20260925 09:29:49.329998 25050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.323356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:50989 (local address 127.10.114.66:38941)
0925 09:29:49.323691 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:49.323697 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:49.324344 (+ 647us) server_negotiation.cc:407] Connection header received
0925 09:29:49.324583 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:49.324587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:49.324643 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:49.324742 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:29:49.325495 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.326279 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.328181 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.328994 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.329184 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:49.329264 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:49.329577 (+ 313us) server_negotiation.cc:299] Negotiation successful
0925 09:29:49.329716 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260925 09:29:49.330108 25063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.323284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:38941 (local address 127.10.114.65:50989)
0925 09:29:49.324246 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:49.324262 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:49.324403 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:49.324756 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:49.324760 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:49.324787 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:29:49.325352 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:49.325367 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.326427 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.326431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:49.328018 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:49.328033 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.329247 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.329255 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:49.329420 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:49.329442 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 09:29:49.329543 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 09:29:49.329660 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"mutex_wait_us":41,"thread_start_us":620,"threads_started":1}
I20260925 09:29:49.330228 25062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.323127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:34895 (local address 127.10.114.65:34521)
0925 09:29:49.324052 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:49.324082 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:49.324325 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:49.324687 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:49.324692 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:49.324716 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:29:49.325207 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:49.325221 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.326300 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.326307 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:49.328051 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:49.328065 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.329210 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.329217 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:49.329407 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:49.329438 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 09:29:49.329552 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 09:29:49.329677 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":88,"thread_start_us":465,"threads_started":1}
I20260925 09:29:49.331113 24990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "738ae18afa664f3b94687a2e95225ada" candidate_uuid: "8df6aef7bde44bf39163ca13c1810d6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca4dc01101f74ee79542718e10cf7df0" is_pre_election: true
I20260925 09:29:49.331115 24857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "738ae18afa664f3b94687a2e95225ada" candidate_uuid: "8df6aef7bde44bf39163ca13c1810d6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3193e95317f24e62a1d32918e04e927f" is_pre_election: true
I20260925 09:29:49.331558 24990 raft_consensus.cc:2454] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df6aef7bde44bf39163ca13c1810d6b in term 0.
I20260925 09:29:49.331602 24857 raft_consensus.cc:2454] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df6aef7bde44bf39163ca13c1810d6b in term 0.
I20260925 09:29:49.332240 24658 leader_election.cc:302] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df6aef7bde44bf39163ca13c1810d6b, ca4dc01101f74ee79542718e10cf7df0; no voters:
I20260925 09:29:49.332677 25060 raft_consensus.cc:2781] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:29:49.332844 25060 raft_consensus.cc:482] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:29:49.332958 25060 raft_consensus.cc:3037] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:29:49.335364 25060 raft_consensus.cc:504] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } } peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } }
I20260925 09:29:49.336053 25060 leader_election.cc:288] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [CANDIDATE]: Term 1 election: Requested vote from peers 3193e95317f24e62a1d32918e04e927f (127.10.114.66:38941), ca4dc01101f74ee79542718e10cf7df0 (127.10.114.67:34895)
I20260925 09:29:49.336473 24990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "738ae18afa664f3b94687a2e95225ada" candidate_uuid: "8df6aef7bde44bf39163ca13c1810d6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca4dc01101f74ee79542718e10cf7df0"
I20260925 09:29:49.336468 24857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "738ae18afa664f3b94687a2e95225ada" candidate_uuid: "8df6aef7bde44bf39163ca13c1810d6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3193e95317f24e62a1d32918e04e927f"
I20260925 09:29:49.336645 24990 raft_consensus.cc:3037] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:29:49.336668 24857 raft_consensus.cc:3037] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:29:49.338644 24990 raft_consensus.cc:2454] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df6aef7bde44bf39163ca13c1810d6b in term 1.
I20260925 09:29:49.338775 24857 raft_consensus.cc:2454] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df6aef7bde44bf39163ca13c1810d6b in term 1.
I20260925 09:29:49.339047 24658 leader_election.cc:302] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df6aef7bde44bf39163ca13c1810d6b, ca4dc01101f74ee79542718e10cf7df0; no voters:
I20260925 09:29:49.339432 25060 raft_consensus.cc:2781] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:29:49.339900 25060 raft_consensus.cc:686] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [term 1 LEADER]: Becoming Leader. State: Replica: 8df6aef7bde44bf39163ca13c1810d6b, State: Running, Role: LEADER
I20260925 09:29:49.340363 25060 consensus_queue.cc:237] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } } peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } }
I20260925 09:29:49.344666 24577 catalog_manager.cc:5495] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df6aef7bde44bf39163ca13c1810d6b (127.10.114.65). New cstate: current_term: 1 leader_uuid: "8df6aef7bde44bf39163ca13c1810d6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } health_report { overall_health: UNKNOWN } } }
I20260925 09:29:49.397876 25051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.392363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33622 (local address 127.10.114.65:46239)
0925 09:29:49.392554 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:49.392561 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:49.392647 (+ 86us) server_negotiation.cc:407] Connection header received
0925 09:29:49.392880 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:49.392884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:49.392956 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:49.393102 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:29:49.393466 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.394142 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.395637 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.396116 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.396254 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:49.396838 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:29:49.397329 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:29:49.397486 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:49.397538 (+ 52us) server_negotiation.cc:299] Negotiation successful
0925 09:29:49.397615 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:29:49.405874 24990 raft_consensus.cc:1261] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0 [term 1 FOLLOWER]: Refusing update from remote peer 8df6aef7bde44bf39163ca13c1810d6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:29:49.405920 24857 raft_consensus.cc:1261] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f [term 1 FOLLOWER]: Refusing update from remote peer 8df6aef7bde44bf39163ca13c1810d6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:29:49.406836 25064 consensus_queue.cc:1035] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:29:49.407161 25060 consensus_queue.cc:1035] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:29:49.420934 25067 mvcc.cc:205] Tried to move back new op lower bound from 7333185902195912704 to 7333185901940854784. Current Snapshot: MvccSnapshot[applied={T|T < 7333185902195912704 or (T in {7333185902195912704})}]
I20260925 09:29:49.421696 25068 mvcc.cc:205] Tried to move back new op lower bound from 7333185902195912704 to 7333185901940854784. Current Snapshot: MvccSnapshot[applied={T|T < 7333185902195912704}]
I20260925 09:29:49.431905 24577 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:29:49.432149 24577 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:29:49.435348 24577 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169: Sending DeleteTablet for 3 replicas of tablet 738ae18afa664f3b94687a2e95225ada
I20260925 09:29:49.436403 24703 tablet_service.cc:1514] Processing DeleteTablet for tablet 738ae18afa664f3b94687a2e95225ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:29:49 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:33608
I20260925 09:29:49.436811 24837 tablet_service.cc:1514] Processing DeleteTablet for tablet 738ae18afa664f3b94687a2e95225ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:29:49 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:60632
I20260925 09:29:49.436856 24970 tablet_service.cc:1514] Processing DeleteTablet for tablet 738ae18afa664f3b94687a2e95225ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:29:49 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:40304
I20260925 09:29:49.437686 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 24640
I20260925 09:29:49.447679 25077 tablet_replica.cc:330] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f: stopping tablet replica
I20260925 09:29:49.448822 25078 tablet_replica.cc:330] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0: stopping tablet replica
I20260925 09:29:49.449379 25078 raft_consensus.cc:2229] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:29:49.460114 25077 raft_consensus.cc:2229] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:29:49.460703 25078 raft_consensus.cc:2258] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:29:49.461228 25077 raft_consensus.cc:2258] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f [term 1 FOLLOWER]: Raft consensus is shut down!
Sep 25 09:29:49 dist-test-slave-9pj8 krb5kdc[24534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:29:49.464134 25078 ts_tablet_manager.cc:1905] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:29:49.466395 25076 tablet_replica.cc:330] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b: stopping tablet replica
I20260925 09:29:49.466861 25076 raft_consensus.cc:2229] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:29:49.467574 25076 raft_consensus.cc:2258] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:29:49.465967 25077 ts_tablet_manager.cc:1905] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:29:49.471046 25051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.439028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33626 (local address 127.10.114.65:46239)
0925 09:29:49.439235 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:49.439244 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:49.448252 (+ 9008us) server_negotiation.cc:407] Connection header received
0925 09:29:49.448448 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:49.448456 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:49.448583 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:49.448708 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:49.451549 (+ 2841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.453110 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.455877 (+ 2767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.456880 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.457010 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:49.463824 (+ 6814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:49.463837 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:49.463862 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:49.463910 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:49.467729 (+ 3819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:49.468620 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:49.468628 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:49.468631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:49.468810 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:49.469244 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:49.469249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:49.469251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:49.469672 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:49.469939 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:49.470465 (+ 526us) server_negotiation.cc:299] Negotiation successful
0925 09:29:49.470681 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 09:29:49.472481 24763 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:29:49.476830 25077 ts_tablet_manager.cc:1918] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:29:49.477030 25077 log.cc:1196] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/738ae18afa664f3b94687a2e95225ada
I20260925 09:29:49.477511 25077 ts_tablet_manager.cc:1939] T 738ae18afa664f3b94687a2e95225ada P 3193e95317f24e62a1d32918e04e927f: Deleting consensus metadata
I20260925 09:29:49.479051 24563 catalog_manager.cc:4841] TS 3193e95317f24e62a1d32918e04e927f (127.10.114.66:38941): tablet 738ae18afa664f3b94687a2e95225ada (table test-table [id=85645259b5ae48b5806505c2d961b718]) successfully deleted
I20260925 09:29:49.480336 25078 ts_tablet_manager.cc:1918] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:29:49.480492 25078 log.cc:1196] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/738ae18afa664f3b94687a2e95225ada
I20260925 09:29:49.480990 25078 ts_tablet_manager.cc:1939] T 738ae18afa664f3b94687a2e95225ada P ca4dc01101f74ee79542718e10cf7df0: Deleting consensus metadata
I20260925 09:29:49.482712 24565 catalog_manager.cc:4841] TS ca4dc01101f74ee79542718e10cf7df0 (127.10.114.67:34895): tablet 738ae18afa664f3b94687a2e95225ada (table test-table [id=85645259b5ae48b5806505c2d961b718]) successfully deleted
I20260925 09:29:49.570094 25076 ts_tablet_manager.cc:1905] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:29:49.572474 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 24640
W20260925 09:29:49.591178 24566 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.114.65:46239 (error 108)
I20260925 09:29:49.592013 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 24774
I20260925 09:29:49.593114 25082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.592323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:46239 (local address 127.0.0.1:33634)
0925 09:29:49.592808 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:49.592965 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":241,"threads_started":1}
W20260925 09:29:49.593369 24566 catalog_manager.cc:4584] TS 8df6aef7bde44bf39163ca13c1810d6b (127.10.114.65:46239): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 738ae18afa664f3b94687a2e95225ada: Network error: Client connection negotiation failed: client connection to 127.10.114.65:46239: connect: Connection refused (error 111)
Sep 25 09:29:49 dist-test-slave-9pj8 krb5kdc[24534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.66@KRBTEST.COM
I20260925 09:29:49.606700 25050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.592825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60646 (local address 127.10.114.66:38941)
0925 09:29:49.592985 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:49.592992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:49.593033 (+ 41us) server_negotiation.cc:407] Connection header received
0925 09:29:49.593249 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:49.593255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:49.593349 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:49.593471 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:49.594762 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.595439 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.596867 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.597235 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.597325 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:49.601337 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:49.601363 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:49.601383 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:49.601429 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:49.604203 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:49.604813 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:49.604825 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:49.604833 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:49.604960 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:49.605308 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:49.605311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:49.605313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:49.605681 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:49.605922 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:49.606109 (+ 187us) server_negotiation.cc:299] Negotiation successful
0925 09:29:49.606313 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:29:49.607321 24897 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:29:49.647109 25082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.646651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:46239 (local address 127.0.0.1:33646)
0925 09:29:49.646853 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:49.646959 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260925 09:29:49.648162 25082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.647871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:46239 (local address 127.0.0.1:33656)
0925 09:29:49.647987 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:49.648049 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260925 09:29:49.693878 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 24774
I20260925 09:29:49.708514 25082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.708079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:46239 (local address 127.0.0.1:33666)
0925 09:29:49.708281 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:49.708371 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260925 09:29:49.709506 25082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.709158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:46239 (local address 127.0.0.1:33674)
0925 09:29:49.709301 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:49.709381 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260925 09:29:49.710729 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 24907
Sep 25 09:29:49 dist-test-slave-9pj8 krb5kdc[24534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.67@KRBTEST.COM
I20260925 09:29:49.725869 25052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.711432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40312 (local address 127.10.114.67:34895)
0925 09:29:49.711598 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:49.711603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:49.711621 (+ 18us) server_negotiation.cc:407] Connection header received
0925 09:29:49.711703 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:49.711706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:49.711830 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:49.711918 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:49.713189 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.713845 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.715102 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:49.715467 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:49.715551 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:49.720387 (+ 4836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:49.720400 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:49.720415 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:49.720443 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:49.723038 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:49.723707 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:49.723718 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:49.723722 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:49.723984 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:49.724362 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:49.724368 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:49.724371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:49.724707 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:49.725005 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:49.725337 (+ 332us) server_negotiation.cc:299] Negotiation successful
0925 09:29:49.725544 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 09:29:49.726466 25030 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:29:49.790026 25082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.789570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:46239 (local address 127.0.0.1:33678)
0925 09:29:49.789793 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:49.789881 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260925 09:29:49.791077 25082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:49.790693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:46239 (local address 127.0.0.1:33692)
0925 09:29:49.790881 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:49.790955 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260925 09:29:49.812765 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 24907
I20260925 09:29:49.829847 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 24547
I20260925 09:29:49.830940 24609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:29:49.942739 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 24547
I20260925 09:29:49.963434 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:38571
--webserver_interface=127.10.114.126
--webserver_port=37863
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:38677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:38571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:29:50.276154 25086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:50.276484 25086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:50.276639 25086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:50.283721 25086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:29:50.283816 25086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:50.283885 25086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:50.283957 25086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:29:50.284029 25086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:29:50.292956 25086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:38571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:38571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=37863
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:50.294104 25086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:50.295328 25086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:50.317011 25092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:50.317022 25093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:50.317013 25095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:50.317806 25086 server_base.cc:1034] running on GCE node
I20260925 09:29:50.318611 25086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:50.320391 25086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:50.321587 25086 hybrid_clock.cc:648] HybridClock initialized: now 1790328590321529 us; error 55 us; skew 500 ppm
I20260925 09:29:50.321973 25086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:50 dist-test-slave-9pj8 krb5kdc[24534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328590, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:50.325601 25086 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:29:50.327224 25086 webserver.cc:460] Webserver started at http://127.10.114.126:37863/ using document root <none> and password file <none>
I20260925 09:29:50.328035 25086 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:50.328192 25086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:29:50.333860 25086 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:29:50.336818 25102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:50.338114 25086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:29:50.338290 25086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "d81c17443e4c4671b82396246ea49169"
format_stamp: "Formatted at 2026-09-25 09:29:47 on dist-test-slave-9pj8"
server_key: "5169b08cfd8670a31beccd733fb375fc"
server_key_iv: "34b908e1780aa450eefd3c4afd2bb2ed"
server_key_version: "encryptionkey@0"
I20260925 09:29:50.339357 25086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:50.365042 25086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:50.369335 25086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:50.369603 25086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:50.390223 25086 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:38571
I20260925 09:29:50.390246 25153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:38571 every 8 connection(s)
I20260925 09:29:50.391829 25086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:29:50.393464 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 25086
I20260925 09:29:50.394460 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:46239
--local_ip_for_outbound_sockets=127.10.114.65
--tserver_master_addrs=127.10.114.126:38571
--webserver_port=34325
--webserver_interface=127.10.114.65
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:38677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:29:50.401719 25154 sys_catalog.cc:263] Verifying existing consensus state
I20260925 09:29:50.405411 25154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169: Bootstrap starting.
I20260925 09:29:50.424870 25154 log.cc:824] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169: Log is configured to *not* fsync() on all Append() calls
I20260925 09:29:50.436625 25154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 09:29:50.437140 25154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169: Bootstrap complete.
I20260925 09:29:50.446151 25154 raft_consensus.cc:348] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81c17443e4c4671b82396246ea49169" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 38571 } }
I20260925 09:29:50.446935 25154 raft_consensus.cc:729] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d81c17443e4c4671b82396246ea49169, State: Initialized, Role: FOLLOWER
I20260925 09:29:50.447546 25154 consensus_queue.cc:260] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81c17443e4c4671b82396246ea49169" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 38571 } }
I20260925 09:29:50.447846 25154 raft_consensus.cc:388] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:29:50.447984 25154 raft_consensus.cc:482] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:29:50.448169 25154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:29:50.451681 25154 raft_consensus.cc:504] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81c17443e4c4671b82396246ea49169" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 38571 } }
I20260925 09:29:50.452252 25154 leader_election.cc:302] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d81c17443e4c4671b82396246ea49169; no voters:
I20260925 09:29:50.452850 25154 leader_election.cc:288] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 09:29:50.452971 25158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:29:50.454200 25158 raft_consensus.cc:686] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 2 LEADER]: Becoming Leader. State: Replica: d81c17443e4c4671b82396246ea49169, State: Running, Role: LEADER
I20260925 09:29:50.454792 25158 consensus_queue.cc:237] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81c17443e4c4671b82396246ea49169" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 38571 } }
I20260925 09:29:50.455157 25154 sys_catalog.cc:564] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:29:50.457275 25160 sys_catalog.cc:455] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d81c17443e4c4671b82396246ea49169. Latest consensus state: current_term: 2 leader_uuid: "d81c17443e4c4671b82396246ea49169" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81c17443e4c4671b82396246ea49169" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 38571 } } }
I20260925 09:29:50.457619 25160 sys_catalog.cc:458] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [sys.catalog]: This master's current role is: LEADER
I20260925 09:29:50.457314 25159 sys_catalog.cc:455] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d81c17443e4c4671b82396246ea49169" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81c17443e4c4671b82396246ea49169" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 38571 } } }
I20260925 09:29:50.457718 25159 sys_catalog.cc:458] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [sys.catalog]: This master's current role is: LEADER
I20260925 09:29:50.463147 25164 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:29:50.470928 25164 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:29:50.472332 25164 catalog_manager.cc:1259] Loaded cluster ID: fd8036fd223d41dc910de92db291d028
I20260925 09:29:50.472489 25164 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:29:50.475593 25164 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260925 09:29:50.494518 25164 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260925 09:29:50.510430 25164 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:29:50.511965 25164 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169: Loaded TSK: 0
I20260925 09:29:50.513083 25164 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 09:29:50.750972 25156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:50.751228 25156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:50.751372 25156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:50.758633 25156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:50.758728 25156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:50.758886 25156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:29:50.768111 25156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:46239
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=34325
--tserver_master_addrs=127.10.114.126:38571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:50.769332 25156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:50.770617 25156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:50.785411 25181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:50.785447 25182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:50.786644 25184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:50.786837 25156 server_base.cc:1034] running on GCE node
I20260925 09:29:50.787530 25156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:50.788677 25156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:50.789915 25156 hybrid_clock.cc:648] HybridClock initialized: now 1790328590789853 us; error 53 us; skew 500 ppm
I20260925 09:29:50.790273 25156 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:50 dist-test-slave-9pj8 krb5kdc[24534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328590, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:50.793622 25156 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:29:50.795222 25156 webserver.cc:460] Webserver started at http://127.10.114.65:34325/ using document root <none> and password file <none>
I20260925 09:29:50.795991 25156 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:50.796173 25156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:29:50.801800 25156 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:29:50.804944 25191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:50.806380 25156 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:29:50.806550 25156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "8df6aef7bde44bf39163ca13c1810d6b"
format_stamp: "Formatted at 2026-09-25 09:29:47 on dist-test-slave-9pj8"
server_key: "2417cd28b83f1c8ad7ad507ddd28e94d"
server_key_iv: "4b3c54f0dd36291ec0c1a5ce5e96b79a"
server_key_version: "encryptionkey@0"
I20260925 09:29:50.807523 25156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:50.823828 25156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:50.827863 25156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:50.828133 25156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:50.829121 25156 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:29:50.830989 25198 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260925 09:29:50.836440 25156 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 09:29:50.836530 25156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260925 09:29:50.836692 25156 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 09:29:50.839870 25156 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 09:29:50.839959 25156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260925 09:29:50.840317 25198 tablet_bootstrap.cc:492] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b: Bootstrap starting.
I20260925 09:29:50.850332 25198 log.cc:824] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b: Log is configured to *not* fsync() on all Append() calls
I20260925 09:29:50.867902 25198 tablet_bootstrap.cc:492] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 09:29:50.868916 25198 tablet_bootstrap.cc:492] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b: Bootstrap complete.
I20260925 09:29:50.869536 25198 ts_tablet_manager.cc:1397] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b: Time spent bootstrapping tablet: real 0.030s user 0.019s sys 0.008s
I20260925 09:29:50.883824 25198 raft_consensus.cc:348] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } } peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } }
I20260925 09:29:50.885407 25198 raft_consensus.cc:729] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df6aef7bde44bf39163ca13c1810d6b, State: Initialized, Role: FOLLOWER
I20260925 09:29:50.886343 25198 consensus_queue.cc:260] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } } peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } }
I20260925 09:29:50.888579 25198 ts_tablet_manager.cc:1428] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b: Time spent starting tablet: real 0.019s user 0.013s sys 0.006s
I20260925 09:29:50.888751 25156 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:46239
I20260925 09:29:50.888787 25306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:46239 every 8 connection(s)
I20260925 09:29:50.889942 25156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:29:50.899986 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 25156
I20260925 09:29:50.900719 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:38941
--local_ip_for_outbound_sockets=127.10.114.66
--tserver_master_addrs=127.10.114.126:38571
--webserver_port=36151
--webserver_interface=127.10.114.66
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:38677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:29:50 dist-test-slave-9pj8 krb5kdc[24534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328590, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:29:50.911595 25309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:50.892432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.65:46851)
0925 09:29:50.893242 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:50.893286 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:50.893658 (+ 372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:50.894827 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:50.894834 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:50.895370 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:50.896010 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:50.896028 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:50.897755 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:50.897759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:50.898957 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:50.898968 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:50.899979 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:50.899985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:50.900271 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:50.901145 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:50.901181 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:50.903860 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:50.907310 (+ 3450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:50.907319 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:50.907332 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:50.907873 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:50.908358 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:50.908363 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:50.908367 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:50.908555 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:50.909245 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:50.909252 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:50.909570 (+ 318us) client_negotiation.cc:769] Sending connection context
0925 09:29:50.909834 (+ 264us) client_negotiation.cc:240] Negotiation successful
0925 09:29:50.910342 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":360,"threads_started":1}
I20260925 09:29:50.911615 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:50.892560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:46851 (local address 127.10.114.126:38571)
0925 09:29:50.893223 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:50.893238 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:50.893403 (+ 165us) server_negotiation.cc:407] Connection header received
0925 09:29:50.894096 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:50.894109 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:50.894425 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:50.894724 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:50.896221 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:50.897447 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:50.899091 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:50.899712 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:50.899919 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:50.904110 (+ 4191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:50.904117 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:50.904129 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:50.904163 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:50.907061 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:50.908042 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:50.908048 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:50.908053 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:50.908241 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:50.908674 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:50.908683 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:50.908685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:50.909103 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:50.909315 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:50.910066 (+ 751us) server_negotiation.cc:299] Negotiation successful
0925 09:29:50.910432 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":301,"threads_started":1}
I20260925 09:29:50.914958 25307 heartbeater.cc:347] Connected to a master server at 127.10.114.126:38571
I20260925 09:29:50.915249 25307 heartbeater.cc:464] Registering TS with master...
I20260925 09:29:50.915966 25307 heartbeater.cc:511] Master 127.10.114.126:38571 requested a full tablet report, sending...
I20260925 09:29:50.919414 25119 ts_manager.cc:194] Registered new tserver with Master: 8df6aef7bde44bf39163ca13c1810d6b (127.10.114.65:46239)
I20260925 09:29:50.926359 25119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:46851
Sep 25 09:29:50 dist-test-slave-9pj8 krb5kdc[24534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328590, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:29:50.940894 25312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:50.924275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:46239 (local address 127.0.0.1:33704)
0925 09:29:50.925095 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:50.925139 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:50.925311 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:50.926568 (+ 1257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:50.926577 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:50.927611 (+ 1034us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:50.927993 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:50.928008 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:50.929582 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:50.929588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:50.930562 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:50.930574 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:50.931479 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:50.931485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:50.931697 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:50.932443 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:50.932461 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:50.934516 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:50.938003 (+ 3487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:50.938015 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:50.938022 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:50.938413 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:50.938853 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:50.938859 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:50.938863 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:50.939029 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:50.939665 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:50.939675 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:50.939896 (+ 221us) client_negotiation.cc:769] Sending connection context
0925 09:29:50.940091 (+ 195us) client_negotiation.cc:240] Negotiation successful
0925 09:29:50.940418 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":414,"threads_started":1}
I20260925 09:29:50.940894 25313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:50.924341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33704 (local address 127.10.114.65:46239)
0925 09:29:50.925926 (+ 1585us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:50.925935 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:50.925966 (+ 31us) server_negotiation.cc:407] Connection header received
0925 09:29:50.926055 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:50.926062 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:50.926304 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:50.927541 (+ 1237us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:50.928230 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:50.929365 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:50.930731 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:50.931279 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:50.931436 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:50.934763 (+ 3327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:50.934773 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:50.934784 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:50.934819 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:50.937855 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:50.938556 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:50.938571 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:50.938574 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:50.938724 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:50.939141 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:50.939148 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:50.939171 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:50.939514 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:50.939721 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:50.940230 (+ 509us) server_negotiation.cc:299] Negotiation successful
0925 09:29:50.940494 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":1422,"thread_start_us":1251,"threads_started":1}
I20260925 09:29:50.942497 25241 tablet_service.cc:1514] Processing DeleteTablet for tablet 738ae18afa664f3b94687a2e95225ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:29:49 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:33704
I20260925 09:29:50.944842 25314 tablet_replica.cc:330] stopping tablet replica
I20260925 09:29:50.945238 25314 raft_consensus.cc:2229] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:29:50.945667 25314 raft_consensus.cc:2258] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:29:50.947135 25314 ts_tablet_manager.cc:1905] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:29:50.957401 25314 ts_tablet_manager.cc:1918] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:29:50.957579 25314 log.cc:1196] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/738ae18afa664f3b94687a2e95225ada
I20260925 09:29:50.958148 25314 ts_tablet_manager.cc:1939] T 738ae18afa664f3b94687a2e95225ada P 8df6aef7bde44bf39163ca13c1810d6b: Deleting consensus metadata
I20260925 09:29:50.960747 25106 catalog_manager.cc:4841] TS 8df6aef7bde44bf39163ca13c1810d6b (127.10.114.65:46239): tablet 738ae18afa664f3b94687a2e95225ada (table test-table [id=85645259b5ae48b5806505c2d961b718]) successfully deleted
W20260925 09:29:51.222563 25311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:51.222821 25311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:51.222980 25311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:51.229912 25311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:51.230000 25311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:51.230197 25311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:29:51.239292 25311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:38941
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=36151
--tserver_master_addrs=127.10.114.126:38571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:51.240491 25311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:51.241732 25311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:51.254365 25322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:51.254320 25319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:51.254407 25320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:51.255540 25311 server_base.cc:1034] running on GCE node
I20260925 09:29:51.256137 25311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:51.257019 25311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:51.258203 25311 hybrid_clock.cc:648] HybridClock initialized: now 1790328591258152 us; error 58 us; skew 500 ppm
I20260925 09:29:51.258448 25311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:51 dist-test-slave-9pj8 krb5kdc[24534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328591, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:51.261456 25311 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:29:51.262836 25311 webserver.cc:460] Webserver started at http://127.10.114.66:36151/ using document root <none> and password file <none>
I20260925 09:29:51.263469 25311 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:51.263612 25311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:29:51.268755 25311 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260925 09:29:51.271224 25329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:51.272500 25311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:29:51.272665 25311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "3193e95317f24e62a1d32918e04e927f"
format_stamp: "Formatted at 2026-09-25 09:29:48 on dist-test-slave-9pj8"
server_key: "b230bb6e140f6fa69b2bbd2b09fdacf9"
server_key_iv: "4dfee59ff957795e9f68c9bb1024c8e3"
server_key_version: "encryptionkey@0"
I20260925 09:29:51.273545 25311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:51.297457 25311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:51.301192 25311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:51.301462 25311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:51.302368 25311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:29:51.303658 25311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:29:51.303812 25311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:51.303936 25311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:29:51.304035 25311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:51.337064 25311 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:38941
I20260925 09:29:51.337141 25441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:38941 every 8 connection(s)
I20260925 09:29:51.338553 25311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:29:51.343602 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 25311
I20260925 09:29:51.344312 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:34895
--local_ip_for_outbound_sockets=127.10.114.67
--tserver_master_addrs=127.10.114.126:38571
--webserver_port=33821
--webserver_interface=127.10.114.67
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:38677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:29:51 dist-test-slave-9pj8 krb5kdc[24534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328591, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:29:51.360332 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:51.341175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:53423 (local address 127.10.114.126:38571)
0925 09:29:51.341357 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:51.341361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:51.342599 (+ 1238us) server_negotiation.cc:407] Connection header received
0925 09:29:51.344717 (+ 2118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:51.344722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:51.344814 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:51.344944 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:51.346355 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:51.347051 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:51.348600 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:51.349006 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:51.349141 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:51.353296 (+ 4155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:51.353306 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:51.353313 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:51.353361 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:51.357177 (+ 3816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:51.357865 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:51.357873 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:51.357880 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:51.358051 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:51.358428 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:51.358436 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:51.358442 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:51.358849 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:51.359020 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:51.359617 (+ 597us) server_negotiation.cc:299] Negotiation successful
0925 09:29:51.359910 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 09:29:51.360881 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:51.341659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.66:53423)
0925 09:29:51.342405 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:51.342490 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:51.344359 (+ 1869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:51.345165 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:51.345173 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:51.345642 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:51.346195 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:51.346210 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:51.347214 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:51.347220 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:51.348457 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:51.348468 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:51.349219 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:51.349226 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:51.349495 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:51.350224 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:51.350255 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:51.353065 (+ 2810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:51.357321 (+ 4256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:51.357327 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:51.357339 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:51.357717 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:51.358154 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:51.358157 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:51.358159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:51.358315 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:51.358985 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:51.358990 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:51.359254 (+ 264us) client_negotiation.cc:769] Sending connection context
0925 09:29:51.359459 (+ 205us) client_negotiation.cc:240] Negotiation successful
0925 09:29:51.359868 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":283,"threads_started":1}
I20260925 09:29:51.362357 25442 heartbeater.cc:347] Connected to a master server at 127.10.114.126:38571
I20260925 09:29:51.362699 25442 heartbeater.cc:464] Registering TS with master...
I20260925 09:29:51.363600 25442 heartbeater.cc:511] Master 127.10.114.126:38571 requested a full tablet report, sending...
I20260925 09:29:51.365341 25119 ts_manager.cc:194] Registered new tserver with Master: 3193e95317f24e62a1d32918e04e927f (127.10.114.66:38941)
I20260925 09:29:51.366698 25119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:53423
W20260925 09:29:51.672986 25445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:51.673236 25445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:51.673331 25445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:51.680452 25445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:51.680549 25445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:51.680656 25445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:29:51.689684 25445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:34895
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=33821
--tserver_master_addrs=127.10.114.126:38571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:51.690778 25445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:51.692070 25445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:51.704537 25451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:51.704587 25450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:51.704566 25453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:51.705806 25445 server_base.cc:1034] running on GCE node
I20260925 09:29:51.706411 25445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:51.707262 25445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:51.708459 25445 hybrid_clock.cc:648] HybridClock initialized: now 1790328591708423 us; error 31 us; skew 500 ppm
I20260925 09:29:51.708750 25445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:51 dist-test-slave-9pj8 krb5kdc[24534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328591, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:51.711755 25445 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:29:51.713179 25445 webserver.cc:460] Webserver started at http://127.10.114.67:33821/ using document root <none> and password file <none>
I20260925 09:29:51.713790 25445 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:51.713924 25445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:29:51.718845 25445 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260925 09:29:51.721510 25460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:51.722868 25445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 09:29:51.723035 25445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "ca4dc01101f74ee79542718e10cf7df0"
format_stamp: "Formatted at 2026-09-25 09:29:48 on dist-test-slave-9pj8"
server_key: "31fb53e9f9f6c2b00d98291319c3b7dd"
server_key_iv: "498f8088bf22e7fc17bd20470912cfc8"
server_key_version: "encryptionkey@0"
I20260925 09:29:51.724011 25445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:51.743459 25445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:51.747113 25445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:51.747371 25445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:51.748333 25445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:29:51.749571 25445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:29:51.749655 25445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:51.749802 25445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:29:51.749882 25445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:51.782999 25445 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:34895
I20260925 09:29:51.783085 25572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:34895 every 8 connection(s)
I20260925 09:29:51.784394 25445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:29:51.791949 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 25445
Sep 25 09:29:51 dist-test-slave-9pj8 krb5kdc[24534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328591, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:29:51.803349 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:51.786687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:47799 (local address 127.10.114.126:38571)
0925 09:29:51.786859 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:51.786863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:51.787962 (+ 1099us) server_negotiation.cc:407] Connection header received
0925 09:29:51.789681 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:51.789687 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:51.789769 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:51.789890 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:51.791239 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:51.792021 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:51.793993 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:51.794406 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:51.794539 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:51.797738 (+ 3199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:51.797746 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:51.797749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:51.797782 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:51.800434 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:51.801075 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:51.801083 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:51.801088 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:51.801245 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:51.801742 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:51.801746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:51.801748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:51.802045 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:51.802197 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:51.802962 (+ 765us) server_negotiation.cc:299] Negotiation successful
0925 09:29:51.803115 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 09:29:51.804649 25575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:51.787085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.67:47799)
0925 09:29:51.787756 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:51.787813 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:51.789424 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:51.790112 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:51.790120 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:51.790563 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:29:51.791069 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:51.791085 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:51.792194 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:51.792201 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:51.793850 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:51.793862 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:51.794624 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:51.794630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:51.794843 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:51.795467 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:29:51.795492 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:29:51.797527 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:29:51.800563 (+ 3036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:51.800571 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:51.800585 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:51.800963 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:51.801377 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:29:51.801384 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:29:51.801391 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:29:51.801624 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:29:51.802174 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:29:51.802179 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:29:51.802561 (+ 382us) client_negotiation.cc:769] Sending connection context
0925 09:29:51.802873 (+ 312us) client_negotiation.cc:240] Negotiation successful
0925 09:29:51.803362 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":271,"threads_started":1}
I20260925 09:29:51.806001 25573 heartbeater.cc:347] Connected to a master server at 127.10.114.126:38571
I20260925 09:29:51.806365 25573 heartbeater.cc:464] Registering TS with master...
I20260925 09:29:51.806770 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:51.792869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48950 (local address 127.10.114.126:38571)
0925 09:29:51.793568 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:51.793576 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:51.793593 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:29:51.793646 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:51.793649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:51.793712 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:51.793820 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:51.795255 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:51.795998 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:51.797360 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:51.797781 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:51.797919 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:51.800669 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:51.800674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:51.800677 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:51.800709 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:51.803858 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:51.804544 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:51.804549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:51.804551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:51.804708 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:51.805215 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:51.805220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:51.805224 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:51.805582 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:51.805824 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:51.806216 (+ 392us) server_negotiation.cc:299] Negotiation successful
0925 09:29:51.806379 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":480,"threads_started":1}
I20260925 09:29:51.807240 25573 heartbeater.cc:511] Master 127.10.114.126:38571 requested a full tablet report, sending...
I20260925 09:29:51.808966 25118 ts_manager.cc:194] Registered new tserver with Master: ca4dc01101f74ee79542718e10cf7df0 (127.10.114.67:34895)
I20260925 09:29:51.809913 25118 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:47799
I20260925 09:29:51.821492 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:29:51.827775 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:51.822283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48954 (local address 127.10.114.126:38571)
0925 09:29:51.822473 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:51.822477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:51.822846 (+ 369us) server_negotiation.cc:407] Connection header received
0925 09:29:51.822988 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:51.822991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:51.823098 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:51.823209 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:29:51.823551 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:51.824244 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:51.825626 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:51.826040 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:51.826135 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:51.826732 (+ 597us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:29:51.827138 (+ 406us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:29:51.827242 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:51.827388 (+ 146us) server_negotiation.cc:299] Negotiation successful
0925 09:29:51.827469 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 09:29:51.828768 25118 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:48954:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:29:51.835577 25118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:29:51.847208 25241 tablet_service.cc:1467] Processing CreateTablet for tablet 407a5274c3b645c2a3cb78d286516581 (DEFAULT_TABLE table=test-table [id=eebc1482256f47da9b70ce3ba6d50372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:29:51.848309 25241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 407a5274c3b645c2a3cb78d286516581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:29:51.861632 25587 tablet_bootstrap.cc:492] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b: Bootstrap starting.
I20260925 09:29:51.863430 25587 tablet_bootstrap.cc:654] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b: Neither blocks nor log segments found. Creating new log.
I20260925 09:29:51.865854 25583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:51.845744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:38941 (local address 127.0.0.1:60662)
0925 09:29:51.857968 (+ 12224us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:51.857989 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:51.858099 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:51.859121 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:51.859124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:51.859143 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:29:51.859363 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:51.859370 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:51.861632 (+ 2262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:51.861638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:51.863139 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:51.863162 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:51.865036 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:51.865042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:51.865200 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:51.865267 (+ 67us) client_negotiation.cc:769] Sending connection context
0925 09:29:51.865376 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 09:29:51.865510 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12092,"thread_start_us":11538,"threads_started":1}
I20260925 09:29:51.865969 25586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:51.846016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60662 (local address 127.10.114.66:38941)
0925 09:29:51.858319 (+ 12303us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:51.858331 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:51.858376 (+ 45us) server_negotiation.cc:407] Connection header received
0925 09:29:51.858503 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:51.858516 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:51.858848 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:51.859145 (+ 297us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:29:51.859551 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:51.861427 (+ 1876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:51.863350 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:51.864844 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:51.865005 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:51.865242 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:51.865442 (+ 200us) server_negotiation.cc:299] Negotiation successful
0925 09:29:51.865619 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12079,"thread_start_us":11898,"threads_started":1}
I20260925 09:29:51.866001 25587 tablet_bootstrap.cc:492] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b: No bootstrap required, opened a new log
I20260925 09:29:51.866161 25587 ts_tablet_manager.cc:1397] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260925 09:29:51.867101 25587 raft_consensus.cc:348] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } } peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } }
I20260925 09:29:51.867537 25587 raft_consensus.cc:374] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:29:51.867663 25587 raft_consensus.cc:729] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df6aef7bde44bf39163ca13c1810d6b, State: Initialized, Role: FOLLOWER
I20260925 09:29:51.868219 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:51.845116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:34895 (local address 127.0.0.1:40322)
0925 09:29:51.857617 (+ 12501us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:51.857639 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:51.859918 (+ 2279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:51.860764 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:51.860769 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:51.860790 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:29:51.861129 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:51.861140 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:51.863005 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:51.863009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:51.865339 (+ 2330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:51.865354 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:51.867420 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:51.867426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:51.867593 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:51.867645 (+ 52us) client_negotiation.cc:769] Sending connection context
0925 09:29:51.867734 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 09:29:51.867885 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12338,"thread_start_us":14891,"threads_started":1}
I20260925 09:29:51.868285 25584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:51.845421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40322 (local address 127.10.114.67:34895)
0925 09:29:51.846704 (+ 1283us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:51.846713 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:51.857771 (+ 11058us) server_negotiation.cc:407] Connection header received
0925 09:29:51.860079 (+ 2308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:51.860098 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:51.860316 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:51.860510 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:29:51.861325 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:51.862833 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:51.865497 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:51.866954 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:51.867070 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:51.867286 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:51.867796 (+ 510us) server_negotiation.cc:299] Negotiation successful
0925 09:29:51.867940 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1079,"thread_start_us":1087,"threads_started":1}
I20260925 09:29:51.868083 25587 consensus_queue.cc:260] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } } peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } }
I20260925 09:29:51.869087 25587 ts_tablet_manager.cc:1428] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 09:29:51.869525 25307 heartbeater.cc:503] Master 127.10.114.126:38571 was elected leader, sending a full tablet report...
I20260925 09:29:51.870213 25377 tablet_service.cc:1467] Processing CreateTablet for tablet 407a5274c3b645c2a3cb78d286516581 (DEFAULT_TABLE table=test-table [id=eebc1482256f47da9b70ce3ba6d50372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:29:51.871678 25377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 407a5274c3b645c2a3cb78d286516581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:29:51.871897 25508 tablet_service.cc:1467] Processing CreateTablet for tablet 407a5274c3b645c2a3cb78d286516581 (DEFAULT_TABLE table=test-table [id=eebc1482256f47da9b70ce3ba6d50372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:29:51.873195 25508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 407a5274c3b645c2a3cb78d286516581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:29:51.884608 25590 tablet_bootstrap.cc:492] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f: Bootstrap starting.
I20260925 09:29:51.884789 25591 tablet_bootstrap.cc:492] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0: Bootstrap starting.
I20260925 09:29:51.888381 25591 tablet_bootstrap.cc:654] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0: Neither blocks nor log segments found. Creating new log.
I20260925 09:29:51.888432 25590 tablet_bootstrap.cc:654] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f: Neither blocks nor log segments found. Creating new log.
I20260925 09:29:51.889341 25591 log.cc:824] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0: Log is configured to *not* fsync() on all Append() calls
I20260925 09:29:51.889664 25590 log.cc:824] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f: Log is configured to *not* fsync() on all Append() calls
I20260925 09:29:51.892204 25590 tablet_bootstrap.cc:492] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f: No bootstrap required, opened a new log
I20260925 09:29:51.892335 25591 tablet_bootstrap.cc:492] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0: No bootstrap required, opened a new log
I20260925 09:29:51.892552 25590 ts_tablet_manager.cc:1397] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260925 09:29:51.892596 25591 ts_tablet_manager.cc:1397] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
W20260925 09:29:51.892668 25308 tablet.cc:2369] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:29:51.899937 25590 raft_consensus.cc:348] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } } peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } }
I20260925 09:29:51.900278 25590 raft_consensus.cc:374] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:29:51.900398 25590 raft_consensus.cc:729] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3193e95317f24e62a1d32918e04e927f, State: Initialized, Role: FOLLOWER
I20260925 09:29:51.900959 25590 consensus_queue.cc:260] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } } peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } }
I20260925 09:29:51.901983 25442 heartbeater.cc:503] Master 127.10.114.126:38571 was elected leader, sending a full tablet report...
I20260925 09:29:51.902556 25590 ts_tablet_manager.cc:1428] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260925 09:29:51.903287 25591 raft_consensus.cc:348] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } } peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } }
I20260925 09:29:51.903813 25591 raft_consensus.cc:374] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:29:51.903959 25591 raft_consensus.cc:729] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca4dc01101f74ee79542718e10cf7df0, State: Initialized, Role: FOLLOWER
I20260925 09:29:51.904567 25591 consensus_queue.cc:260] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } } peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } }
I20260925 09:29:51.905742 25573 heartbeater.cc:503] Master 127.10.114.126:38571 was elected leader, sending a full tablet report...
I20260925 09:29:51.906184 25591 ts_tablet_manager.cc:1428] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
W20260925 09:29:52.035907 25574 tablet.cc:2369] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:29:52.090517 25443 tablet.cc:2369] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:29:52.227979 25594 raft_consensus.cc:482] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:29:52.228401 25594 raft_consensus.cc:504] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } } peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } }
I20260925 09:29:52.229992 25594 leader_election.cc:288] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca4dc01101f74ee79542718e10cf7df0 (127.10.114.67:34895), 8df6aef7bde44bf39163ca13c1810d6b (127.10.114.65:46239)
I20260925 09:29:52.238857 25584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:52.230507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:50761 (local address 127.10.114.67:34895)
0925 09:29:52.230727 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:52.230736 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:52.231510 (+ 774us) server_negotiation.cc:407] Connection header received
0925 09:29:52.231650 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:52.231658 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:52.231740 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:52.231858 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:29:52.232472 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:52.233548 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:52.236510 (+ 2962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:52.237692 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:52.237863 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:52.237932 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:52.238451 (+ 519us) server_negotiation.cc:299] Negotiation successful
0925 09:29:52.238582 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 09:29:52.238973 25597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:52.230508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:57227 (local address 127.10.114.65:46239)
0925 09:29:52.231119 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:52.231126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:52.231238 (+ 112us) server_negotiation.cc:407] Connection header received
0925 09:29:52.231474 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:52.231481 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:52.231604 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:52.231737 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:29:52.232481 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:52.233547 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:52.236497 (+ 2950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:52.237738 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:52.237880 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:52.238055 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:52.238451 (+ 396us) server_negotiation.cc:299] Negotiation successful
0925 09:29:52.238582 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":391,"threads_started":1}
I20260925 09:29:52.239096 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:52.230259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:46239 (local address 127.10.114.66:57227)
0925 09:29:52.231106 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:52.231134 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:52.231333 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:52.231755 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:52.231758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:52.231839 (+ 81us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:29:52.232328 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:52.232339 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:52.233736 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:52.233741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:52.236313 (+ 2572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:52.236322 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:52.237973 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:52.237978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:52.238237 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:52.238321 (+ 84us) client_negotiation.cc:769] Sending connection context
0925 09:29:52.238411 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 09:29:52.238605 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"mutex_wait_us":186,"thread_start_us":458,"threads_started":1}
I20260925 09:29:52.239326 25598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:52.230258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:34895 (local address 127.10.114.66:50761)
0925 09:29:52.231410 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:52.231424 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 09:29:52.231527 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:29:52.231865 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:29:52.231871 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:29:52.231896 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:29:52.232334 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:52.232341 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:52.233736 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:52.233741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:52.236306 (+ 2565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:29:52.236322 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:29:52.237968 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:29:52.237974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:29:52.238227 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:52.238318 (+ 91us) client_negotiation.cc:769] Sending connection context
0925 09:29:52.238411 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 09:29:52.238608 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1035,"mutex_wait_us":812,"thread_start_us":303,"threads_started":1}
I20260925 09:29:52.240109 25261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "407a5274c3b645c2a3cb78d286516581" candidate_uuid: "3193e95317f24e62a1d32918e04e927f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df6aef7bde44bf39163ca13c1810d6b" is_pre_election: true
I20260925 09:29:52.240304 25528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "407a5274c3b645c2a3cb78d286516581" candidate_uuid: "3193e95317f24e62a1d32918e04e927f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca4dc01101f74ee79542718e10cf7df0" is_pre_election: true
I20260925 09:29:52.240547 25261 raft_consensus.cc:2454] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3193e95317f24e62a1d32918e04e927f in term 0.
I20260925 09:29:52.240797 25528 raft_consensus.cc:2454] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3193e95317f24e62a1d32918e04e927f in term 0.
I20260925 09:29:52.241227 25333 leader_election.cc:302] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3193e95317f24e62a1d32918e04e927f, 8df6aef7bde44bf39163ca13c1810d6b; no voters:
I20260925 09:29:52.241614 25594 raft_consensus.cc:2781] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:29:52.241755 25594 raft_consensus.cc:482] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:29:52.241858 25594 raft_consensus.cc:3037] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:29:52.244201 25594 raft_consensus.cc:504] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } } peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } }
I20260925 09:29:52.244709 25594 leader_election.cc:288] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [CANDIDATE]: Term 1 election: Requested vote from peers ca4dc01101f74ee79542718e10cf7df0 (127.10.114.67:34895), 8df6aef7bde44bf39163ca13c1810d6b (127.10.114.65:46239)
I20260925 09:29:52.245119 25528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "407a5274c3b645c2a3cb78d286516581" candidate_uuid: "3193e95317f24e62a1d32918e04e927f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca4dc01101f74ee79542718e10cf7df0"
I20260925 09:29:52.245193 25261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "407a5274c3b645c2a3cb78d286516581" candidate_uuid: "3193e95317f24e62a1d32918e04e927f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df6aef7bde44bf39163ca13c1810d6b"
I20260925 09:29:52.245308 25528 raft_consensus.cc:3037] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:29:52.245431 25261 raft_consensus.cc:3037] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:29:52.247428 25528 raft_consensus.cc:2454] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3193e95317f24e62a1d32918e04e927f in term 1.
I20260925 09:29:52.247664 25261 raft_consensus.cc:2454] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3193e95317f24e62a1d32918e04e927f in term 1.
I20260925 09:29:52.247901 25332 leader_election.cc:302] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3193e95317f24e62a1d32918e04e927f, ca4dc01101f74ee79542718e10cf7df0; no voters:
I20260925 09:29:52.248231 25594 raft_consensus.cc:2781] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:29:52.248622 25594 raft_consensus.cc:686] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [term 1 LEADER]: Becoming Leader. State: Replica: 3193e95317f24e62a1d32918e04e927f, State: Running, Role: LEADER
I20260925 09:29:52.249085 25594 consensus_queue.cc:237] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } } peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } }
I20260925 09:29:52.253656 25119 catalog_manager.cc:5495] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3193e95317f24e62a1d32918e04e927f (127.10.114.66). New cstate: current_term: 1 leader_uuid: "3193e95317f24e62a1d32918e04e927f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3193e95317f24e62a1d32918e04e927f" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 38941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 } health_report { overall_health: UNKNOWN } } }
I20260925 09:29:52.327726 25586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:52.322508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60666 (local address 127.10.114.66:38941)
0925 09:29:52.322675 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:52.322681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:52.322697 (+ 16us) server_negotiation.cc:407] Connection header received
0925 09:29:52.322778 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:52.322781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:52.322851 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:52.322976 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:29:52.323321 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:52.324207 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:52.325570 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:52.325951 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:52.326068 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:52.326551 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:29:52.327146 (+ 595us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:29:52.327267 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:52.327369 (+ 102us) server_negotiation.cc:299] Negotiation successful
0925 09:29:52.327432 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 09:29:52.335366 25261 raft_consensus.cc:1261] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b [term 1 FOLLOWER]: Refusing update from remote peer 3193e95317f24e62a1d32918e04e927f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:29:52.335414 25528 raft_consensus.cc:1261] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0 [term 1 FOLLOWER]: Refusing update from remote peer 3193e95317f24e62a1d32918e04e927f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:29:52.336264 25599 consensus_queue.cc:1035] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df6aef7bde44bf39163ca13c1810d6b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 46239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:29:52.336618 25594 consensus_queue.cc:1035] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca4dc01101f74ee79542718e10cf7df0" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 34895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:29:52.370728 25119 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:48954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:29:52.371150 25119 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:48954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:29:52.375754 25119 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169: Sending DeleteTablet for 3 replicas of tablet 407a5274c3b645c2a3cb78d286516581
I20260925 09:29:52.377105 25241 tablet_service.cc:1514] Processing DeleteTablet for tablet 407a5274c3b645c2a3cb78d286516581 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:29:52 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:33704
I20260925 09:29:52.377079 25508 tablet_service.cc:1514] Processing DeleteTablet for tablet 407a5274c3b645c2a3cb78d286516581 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:29:52 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:40322
I20260925 09:29:52.377398 25377 tablet_service.cc:1514] Processing DeleteTablet for tablet 407a5274c3b645c2a3cb78d286516581 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:29:52 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:60662
I20260925 09:29:52.378383 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 25156
I20260925 09:29:52.379091 25240 tablet_service.cc:1514] Processing DeleteTablet for tablet 407a5274c3b645c2a3cb78d286516581 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:29:52 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:33704
I20260925 09:29:52.379969 25613 tablet_replica.cc:330] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0: stopping tablet replica
I20260925 09:29:52.380419 25613 raft_consensus.cc:2229] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:29:52.380860 25613 raft_consensus.cc:2258] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:29:52.381621 25612 tablet_replica.cc:330] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b: stopping tablet replica
I20260925 09:29:52.381804 25612 raft_consensus.cc:2229] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:29:52.382061 25612 raft_consensus.cc:2258] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:29:52.383155 25612 ts_tablet_manager.cc:1905] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:29:52.384393 25614 tablet_replica.cc:330] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f: stopping tablet replica
I20260925 09:29:52.384816 25614 raft_consensus.cc:2229] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:29:52.385465 25614 raft_consensus.cc:2258] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:29:52.387125 25613 ts_tablet_manager.cc:1905] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:29:52.390084 25614 ts_tablet_manager.cc:1905] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:29:52.391733 25612 ts_tablet_manager.cc:1918] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:29:52.391875 25612 log.cc:1196] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/407a5274c3b645c2a3cb78d286516581
I20260925 09:29:52.392328 25612 ts_tablet_manager.cc:1939] T 407a5274c3b645c2a3cb78d286516581 P 8df6aef7bde44bf39163ca13c1810d6b: Deleting consensus metadata
I20260925 09:29:52.393366 25106 catalog_manager.cc:4841] TS 8df6aef7bde44bf39163ca13c1810d6b (127.10.114.65:46239): tablet 407a5274c3b645c2a3cb78d286516581 (table test-table [id=eebc1482256f47da9b70ce3ba6d50372]) successfully deleted
W20260925 09:29:52.393954 25106 catalog_manager.cc:4808] TS 8df6aef7bde44bf39163ca13c1810d6b (127.10.114.65:46239): delete failed for tablet 407a5274c3b645c2a3cb78d286516581 because the tablet was not found. No further retry: Not found: Tablet not found: 407a5274c3b645c2a3cb78d286516581
I20260925 09:29:52.397040 25614 ts_tablet_manager.cc:1918] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:29:52.397235 25614 log.cc:1196] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/407a5274c3b645c2a3cb78d286516581
I20260925 09:29:52.397771 25614 ts_tablet_manager.cc:1939] T 407a5274c3b645c2a3cb78d286516581 P 3193e95317f24e62a1d32918e04e927f: Deleting consensus metadata
I20260925 09:29:52.398778 25613 ts_tablet_manager.cc:1918] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:29:52.398952 25613 log.cc:1196] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/407a5274c3b645c2a3cb78d286516581
I20260925 09:29:52.399391 25103 catalog_manager.cc:4841] TS 3193e95317f24e62a1d32918e04e927f (127.10.114.66:38941): tablet 407a5274c3b645c2a3cb78d286516581 (table test-table [id=eebc1482256f47da9b70ce3ba6d50372]) successfully deleted
I20260925 09:29:52.399511 25613 ts_tablet_manager.cc:1939] T 407a5274c3b645c2a3cb78d286516581 P ca4dc01101f74ee79542718e10cf7df0: Deleting consensus metadata
I20260925 09:29:52.401207 25105 catalog_manager.cc:4841] TS ca4dc01101f74ee79542718e10cf7df0 (127.10.114.67:34895): tablet 407a5274c3b645c2a3cb78d286516581 (table test-table [id=eebc1482256f47da9b70ce3ba6d50372]) successfully deleted
I20260925 09:29:52.413620 25597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:52.379581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33712 (local address 127.10.114.65:46239)
0925 09:29:52.379815 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:52.379823 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:52.386504 (+ 6681us) server_negotiation.cc:407] Connection header received
0925 09:29:52.386670 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:52.386675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:52.386760 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:52.386862 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:52.393641 (+ 6779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:52.394737 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:52.396679 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:52.397367 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:52.397468 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:52.407075 (+ 9607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:52.407082 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:52.407087 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:52.407128 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:52.410575 (+ 3447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:52.411420 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:52.411425 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:52.411428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:52.411571 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:52.412157 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:52.412164 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:52.412170 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:52.412513 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:52.412771 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:52.413050 (+ 279us) server_negotiation.cc:299] Negotiation successful
0925 09:29:52.413303 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 09:29:52.414314 25301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:29:52.504393 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 25156
I20260925 09:29:52.526794 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 25311
I20260925 09:29:52.540899 25586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:52.527600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60678 (local address 127.10.114.66:38941)
0925 09:29:52.527912 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:52.527918 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:52.527937 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:29:52.528103 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:52.528107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:52.528195 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:52.528308 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:52.529597 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:52.530429 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:52.531701 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:52.532229 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:52.532359 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:52.534752 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:52.534763 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:52.534784 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:52.534820 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:52.537862 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:52.538714 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:52.538725 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:52.538729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:52.538895 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:52.539417 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:52.539423 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:52.539426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:52.539887 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:52.540237 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:52.540366 (+ 129us) server_negotiation.cc:299] Negotiation successful
0925 09:29:52.540555 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260925 09:29:52.541522 25437 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:29:52.650527 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 25311
I20260925 09:29:52.669348 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 25445
I20260925 09:29:52.682798 25584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:52.670127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40324 (local address 127.10.114.67:34895)
0925 09:29:52.670322 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 09:29:52.670328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:29:52.670420 (+ 92us) server_negotiation.cc:407] Connection header received
0925 09:29:52.670656 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:29:52.670660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:29:52.670735 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:29:52.670852 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:29:52.672294 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:52.672963 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:52.674215 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:29:52.674617 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:29:52.674747 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:29:52.677355 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:29:52.677367 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:29:52.677382 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:29:52.677417 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:29:52.680006 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:52.680640 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:52.680650 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:52.680656 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:52.680814 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:29:52.681188 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:29:52.681193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:29:52.681196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:29:52.681594 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:29:52.681911 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0925 09:29:52.682232 (+ 321us) server_negotiation.cc:299] Negotiation successful
0925 09:29:52.682440 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 09:29:52.683506 25568 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:29:52.771520 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 25445
I20260925 09:29:52.788098 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 25086
I20260925 09:29:52.789486 25149 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:29:52.899446 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 25086
I20260925 09:29:52.920567 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:38571
--webserver_interface=127.10.114.126
--webserver_port=37863
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:38677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:38571
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:29:53.244031 25620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:53.244309 25620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:53.244408 25620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:53.251613 25620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:29:53.251713 25620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:53.251763 25620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:53.251899 25620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:29:53.251948 25620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:29:53.261731 25620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:38571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:38571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=37863
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:53.263042 25620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:53.264390 25620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:53.276031 25625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:53.276145 25626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:53.276118 25628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:53.277103 25620 server_base.cc:1034] running on GCE node
I20260925 09:29:53.277879 25620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:53.279285 25620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:53.280493 25620 hybrid_clock.cc:648] HybridClock initialized: now 1790328593280450 us; error 26 us; skew 500 ppm
I20260925 09:29:53.280897 25620 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:53 dist-test-slave-9pj8 krb5kdc[24534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328593, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:53.284296 25620 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:29:53.285966 25620 webserver.cc:460] Webserver started at http://127.10.114.126:37863/ using document root <none> and password file <none>
I20260925 09:29:53.286701 25620 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:53.286893 25620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:29:53.292378 25620 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260925 09:29:53.295107 25635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:53.296454 25620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:29:53.296629 25620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "d81c17443e4c4671b82396246ea49169"
format_stamp: "Formatted at 2026-09-25 09:29:47 on dist-test-slave-9pj8"
server_key: "5169b08cfd8670a31beccd733fb375fc"
server_key_iv: "34b908e1780aa450eefd3c4afd2bb2ed"
server_key_version: "encryptionkey@0"
I20260925 09:29:53.297552 25620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:53.318638 25620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:53.322857 25620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:53.323107 25620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:53.341662 25620 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:38571
I20260925 09:29:53.341702 25686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:38571 every 8 connection(s)
I20260925 09:29:53.343096 25620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:29:53.350142 25687 sys_catalog.cc:263] Verifying existing consensus state
I20260925 09:29:53.351239 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 25620
I20260925 09:29:53.352299 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:46239
--local_ip_for_outbound_sockets=127.10.114.65
--tserver_master_addrs=127.10.114.126:38571
--webserver_port=34325
--webserver_interface=127.10.114.65
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:38677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:29:53.355257 25687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169: Bootstrap starting.
I20260925 09:29:53.377785 25687 log.cc:824] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169: Log is configured to *not* fsync() on all Append() calls
I20260925 09:29:53.393663 25687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 09:29:53.394268 25687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169: Bootstrap complete.
I20260925 09:29:53.404670 25687 raft_consensus.cc:348] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81c17443e4c4671b82396246ea49169" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 38571 } }
I20260925 09:29:53.405560 25687 raft_consensus.cc:729] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d81c17443e4c4671b82396246ea49169, State: Initialized, Role: FOLLOWER
I20260925 09:29:53.406116 25687 consensus_queue.cc:260] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81c17443e4c4671b82396246ea49169" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 38571 } }
I20260925 09:29:53.406363 25687 raft_consensus.cc:388] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:29:53.406491 25687 raft_consensus.cc:482] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:29:53.406674 25687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 2 FOLLOWER]: Advancing to term 3
I20260925 09:29:53.410328 25687 raft_consensus.cc:504] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81c17443e4c4671b82396246ea49169" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 38571 } }
I20260925 09:29:53.410781 25687 leader_election.cc:302] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d81c17443e4c4671b82396246ea49169; no voters:
I20260925 09:29:53.411425 25687 leader_election.cc:288] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 09:29:53.411588 25691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 09:29:53.412776 25691 raft_consensus.cc:686] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [term 3 LEADER]: Becoming Leader. State: Replica: d81c17443e4c4671b82396246ea49169, State: Running, Role: LEADER
I20260925 09:29:53.413326 25691 consensus_queue.cc:237] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81c17443e4c4671b82396246ea49169" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 38571 } }
I20260925 09:29:53.413612 25687 sys_catalog.cc:564] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:29:53.415934 25693 sys_catalog.cc:455] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d81c17443e4c4671b82396246ea49169. Latest consensus state: current_term: 3 leader_uuid: "d81c17443e4c4671b82396246ea49169" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81c17443e4c4671b82396246ea49169" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 38571 } } }
I20260925 09:29:53.416342 25693 sys_catalog.cc:458] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [sys.catalog]: This master's current role is: LEADER
I20260925 09:29:53.416035 25692 sys_catalog.cc:455] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d81c17443e4c4671b82396246ea49169" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81c17443e4c4671b82396246ea49169" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 38571 } } }
I20260925 09:29:53.416458 25692 sys_catalog.cc:458] T 00000000000000000000000000000000 P d81c17443e4c4671b82396246ea49169 [sys.catalog]: This master's current role is: LEADER
I20260925 09:29:53.426614 25697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:29:53.436447 25697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:29:53.438401 25697 catalog_manager.cc:1259] Loaded cluster ID: fd8036fd223d41dc910de92db291d028
I20260925 09:29:53.438565 25697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260925 09:29:53.441260 25697 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fbecf6f89ad google::LogMessage::Fail() at ??:0
@ 0x7fbecf6fb88a google::LogMessage::SendToLog() at ??:0
@ 0x7fbecf6f84de google::LogMessage::Flush() at ??:0
@ 0x7fbecf6f9a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fbee512f512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fbee512c976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fbee51b4610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fbee51b4439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fbed2344345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fbed234791c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fbed2347799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fbed2322eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fbed55fc6db start_thread at ??:0
@ 0x7fbece59261f clone at ??:0
W20260925 09:29:53.762671 25689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:53.762938 25689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:53.763028 25689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:53.769876 25689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:53.769964 25689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:53.770082 25689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:29:53.779389 25689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:46239
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=34325
--tserver_master_addrs=127.10.114.126:38571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:53.780619 25689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:53.781888 25689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:53.794931 25728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:53.794966 25729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:53.795181 25731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:53.796414 25689 server_base.cc:1034] running on GCE node
I20260925 09:29:53.797057 25689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:53.798071 25689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:53.799340 25689 hybrid_clock.cc:648] HybridClock initialized: now 1790328593799274 us; error 47 us; skew 500 ppm
I20260925 09:29:53.799691 25689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:53 dist-test-slave-9pj8 krb5kdc[24534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328593, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:53.802897 25689 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:29:53.804371 25689 webserver.cc:460] Webserver started at http://127.10.114.65:34325/ using document root <none> and password file <none>
I20260925 09:29:53.805076 25689 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:53.805239 25689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:29:53.810676 25689 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260925 09:29:53.813539 25738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:53.814868 25689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260925 09:29:53.815053 25689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "8df6aef7bde44bf39163ca13c1810d6b"
format_stamp: "Formatted at 2026-09-25 09:29:47 on dist-test-slave-9pj8"
server_key: "2417cd28b83f1c8ad7ad507ddd28e94d"
server_key_iv: "4b3c54f0dd36291ec0c1a5ce5e96b79a"
server_key_version: "encryptionkey@0"
I20260925 09:29:53.816066 25689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:53.839991 25689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:53.844098 25689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:53.844374 25689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:53.845340 25689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:29:53.846663 25689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:29:53.846784 25689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:53.846916 25689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:29:53.847005 25689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:53.880950 25689 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:46239
I20260925 09:29:53.881034 25850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:46239 every 8 connection(s)
I20260925 09:29:53.882512 25689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:29:53.886571 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 25689
I20260925 09:29:53.887320 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:38941
--local_ip_for_outbound_sockets=127.10.114.66
--tserver_master_addrs=127.10.114.126:38571
--webserver_port=36151
--webserver_interface=127.10.114.66
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:38677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:29:53.887579 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:53.885563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.65:51683)
0925 09:29:53.886360 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:53.886589 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":317,"threads_started":1}
W20260925 09:29:53.888316 25851 heartbeater.cc:644] Failed to heartbeat to 127.10.114.126:38571 (0 consecutive failures): Network error: Failed to ping master at 127.10.114.126:38571: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
I20260925 09:29:53.889246 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:53.888932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.65:38987)
0925 09:29:53.889076 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:53.889141 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260925 09:29:53.890077 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:53.889766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.65:34259)
0925 09:29:53.889898 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:53.889978 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260925 09:29:53.890296 25851 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 09:29:54.201051 25854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:54.201292 25854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:54.201377 25854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:54.208267 25854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:54.208352 25854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:54.208447 25854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:29:54.217418 25854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:38941
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=36151
--tserver_master_addrs=127.10.114.126:38571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:54.218580 25854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:54.219966 25854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:54.232988 25862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:54.233052 25860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:54.233024 25859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:54.233764 25854 server_base.cc:1034] running on GCE node
I20260925 09:29:54.234369 25854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:54.235255 25854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:54.236496 25854 hybrid_clock.cc:648] HybridClock initialized: now 1790328594236456 us; error 49 us; skew 500 ppm
I20260925 09:29:54.236856 25854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:54 dist-test-slave-9pj8 krb5kdc[24534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328594, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:54.239934 25854 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:29:54.241255 25854 webserver.cc:460] Webserver started at http://127.10.114.66:36151/ using document root <none> and password file <none>
I20260925 09:29:54.241876 25854 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:54.241971 25854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:29:54.247375 25854 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:29:54.249912 25869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:54.251071 25854 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260925 09:29:54.251215 25854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "3193e95317f24e62a1d32918e04e927f"
format_stamp: "Formatted at 2026-09-25 09:29:48 on dist-test-slave-9pj8"
server_key: "b230bb6e140f6fa69b2bbd2b09fdacf9"
server_key_iv: "4dfee59ff957795e9f68c9bb1024c8e3"
server_key_version: "encryptionkey@0"
I20260925 09:29:54.252157 25854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:54.279667 25854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:54.283603 25854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:54.283882 25854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:54.284814 25854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:29:54.285998 25854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:29:54.286078 25854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:54.286160 25854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:29:54.286222 25854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:54.319697 25854 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:38941
I20260925 09:29:54.319778 25981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:38941 every 8 connection(s)
I20260925 09:29:54.321244 25854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:29:54.326013 25984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:54.324309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.66:59385)
0925 09:29:54.324977 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:54.325219 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":251,"threads_started":1}
W20260925 09:29:54.326627 25982 heartbeater.cc:644] Failed to heartbeat to 127.10.114.126:38571 (0 consecutive failures): Network error: Failed to ping master at 127.10.114.126:38571: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
I20260925 09:29:54.327373 25984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:54.327073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.66:34589)
0925 09:29:54.327217 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:54.327277 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 09:29:54.328301 25984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:54.327933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.66:55765)
0925 09:29:54.328078 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:54.328170 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 09:29:54.328505 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 25854
W20260925 09:29:54.328540 25982 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 09:29:54.329075 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:34895
--local_ip_for_outbound_sockets=127.10.114.67
--tserver_master_addrs=127.10.114.126:38571
--webserver_port=33821
--webserver_interface=127.10.114.67
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:38677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:29:54.645012 25985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:29:54.645320 25985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:29:54.645504 25985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:29:54.652436 25985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:29:54.652539 25985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:29:54.652750 25985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:29:54.661856 25985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:38677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:34895
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=33821
--tserver_master_addrs=127.10.114.126:38571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:29:54.662983 25985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:29:54.664292 25985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:29:54.677085 25990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:54.677055 25991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:29:54.677023 25993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:29:54.677892 25985 server_base.cc:1034] running on GCE node
I20260925 09:29:54.678499 25985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:29:54.679382 25985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:29:54.680562 25985 hybrid_clock.cc:648] HybridClock initialized: now 1790328594680509 us; error 50 us; skew 500 ppm
I20260925 09:29:54.680848 25985 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:29:54 dist-test-slave-9pj8 krb5kdc[24534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328594, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:29:54.683862 25985 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:29:54.685196 25985 webserver.cc:460] Webserver started at http://127.10.114.67:33821/ using document root <none> and password file <none>
I20260925 09:29:54.685818 25985 fs_manager.cc:359] Metadata directory not provided
I20260925 09:29:54.685909 25985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:29:54.690815 25985 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260925 09:29:54.693480 26000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:54.694736 25985 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 09:29:54.694857 25985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "ca4dc01101f74ee79542718e10cf7df0"
format_stamp: "Formatted at 2026-09-25 09:29:48 on dist-test-slave-9pj8"
server_key: "31fb53e9f9f6c2b00d98291319c3b7dd"
server_key_iv: "498f8088bf22e7fc17bd20470912cfc8"
server_key_version: "encryptionkey@0"
I20260925 09:29:54.695741 25985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:29:54.712385 25985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:29:54.716517 25985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:29:54.716753 25985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:29:54.717712 25985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:29:54.719092 25985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:29:54.719183 25985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:54.719267 25985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:29:54.719322 25985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:29:54.751494 25985 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:34895
I20260925 09:29:54.751572 26112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:34895 every 8 connection(s)
I20260925 09:29:54.752889 25985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:29:54.757613 26115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:54.755642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.67:55109)
0925 09:29:54.756514 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:54.756764 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":402,"threads_started":1}
W20260925 09:29:54.758165 26113 heartbeater.cc:644] Failed to heartbeat to 127.10.114.126:38571 (0 consecutive failures): Network error: Failed to ping master at 127.10.114.126:38571: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
I20260925 09:29:54.758846 26115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:54.758550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.67:37773)
0925 09:29:54.758673 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:54.758734 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 09:29:54.759157 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 25985
I20260925 09:29:54.759665 26115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:54.759359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.67:52681)
0925 09:29:54.759490 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:54.759565 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260925 09:29:54.759943 26113 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 09:29:54.891703 26117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:54.890805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.65:47907)
0925 09:29:54.891317 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:54.891431 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":270,"threads_started":1}
I20260925 09:29:55.329864 26118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:55.329051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.66:40193)
0925 09:29:55.329513 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:55.329626 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":241,"threads_started":1}
I20260925 09:29:55.761214 26119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:55.760381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.67:57277)
0925 09:29:55.760827 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:55.760957 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":214,"threads_started":1}
I20260925 09:29:55.893385 26120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:55.892606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.65:52327)
0925 09:29:55.893090 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:55.893194 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":266,"threads_started":1}
I20260925 09:29:56.331413 26121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:56.330745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.66:33789)
0925 09:29:56.331182 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:56.331259 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":223,"threads_started":1}
I20260925 09:29:56.762835 26122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:56.762082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.67:47779)
0925 09:29:56.762555 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:56.762663 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":266,"threads_started":1}
I20260925 09:29:56.895180 26123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:56.894416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.65:36897)
0925 09:29:56.894935 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:56.895013 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":304,"threads_started":1}
I20260925 09:29:57.332863 26124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:57.332251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.66:33767)
0925 09:29:57.332632 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:57.332709 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":201,"threads_started":1}
I20260925 09:29:57.764513 26125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:57.763706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.67:57115)
0925 09:29:57.764284 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:57.764356 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":376,"threads_started":1}
I20260925 09:29:57.896864 26126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:57.896066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.65:42951)
0925 09:29:57.896588 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:57.896671 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":258,"threads_started":1}
I20260925 09:29:58.334357 26127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:58.333685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.66:55257)
0925 09:29:58.334096 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:58.334175 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":212,"threads_started":1}
I20260925 09:29:58.766006 26128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:58.765321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.67:40355)
0925 09:29:58.765753 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:58.765829 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":221,"threads_started":1}
I20260925 09:29:58.898509 26129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:58.897766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.65:41455)
0925 09:29:58.898209 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:58.898286 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":219,"threads_started":1}
I20260925 09:29:59.336033 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:59.335244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.66:41419)
0925 09:29:59.335711 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:59.335863 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":235,"threads_started":1}
I20260925 09:29:59.767562 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:59.766853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.67:46951)
0925 09:29:59.767338 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:59.767414 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":254,"threads_started":1}
I20260925 09:29:59.900177 26132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:29:59.899459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.65:53071)
0925 09:29:59.899933 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0925 09:29:59.900023 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":212,"threads_started":1}
I20260925 09:30:00.337632 26133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:00.336982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.66:48575)
0925 09:30:00.337397 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:00.337478 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":227,"threads_started":1}
I20260925 09:30:00.769032 26134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:00.768379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.67:59229)
0925 09:30:00.768798 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:00.768883 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":212,"threads_started":1}
I20260925 09:30:00.901880 26135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:00.901098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.65:50501)
0925 09:30:00.901584 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:00.901679 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":219,"threads_started":1}
I20260925 09:30:01.339497 26136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:01.338659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.66:45995)
0925 09:30:01.339149 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:01.339288 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":225,"threads_started":1}
I20260925 09:30:01.770754 26137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:01.769942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.67:37033)
0925 09:30:01.770424 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:01.770503 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":223,"threads_started":1}
I20260925 09:30:01.903393 26138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:01.902736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.65:54371)
0925 09:30:01.903157 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:01.903252 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":237,"threads_started":1}
I20260925 09:30:02.341070 26139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:02.340406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.66:60347)
0925 09:30:02.340844 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:02.340919 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":231,"threads_started":1}
I20260925 09:30:02.772341 26140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:02.771631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.67:60897)
0925 09:30:02.772093 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:02.772169 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":275,"threads_started":1}
I20260925 09:30:02.905055 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:02.904287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.65:58179)
0925 09:30:02.904799 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:02.904879 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":276,"threads_started":1}
I20260925 09:30:03.342519 26142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:03.341838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.66:52129)
0925 09:30:03.342275 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:03.342371 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":202,"threads_started":1}
I20260925 09:30:03.773891 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:03.773184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.67:36929)
0925 09:30:03.773645 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:03.773728 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":230,"threads_started":1}
I20260925 09:30:03.906716 26144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:03.905950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.65:41949)
0925 09:30:03.906394 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:03.906472 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":234,"threads_started":1}
I20260925 09:30:04.344125 26145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:04.343380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.66:55915)
0925 09:30:04.343894 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:04.343970 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":269,"threads_started":1}
I20260925 09:30:04.775362 26146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:04.774705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.67:52483)
0925 09:30:04.775112 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:04.775211 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":226,"threads_started":1}
I20260925 09:30:04.908531 26147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:04.907856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.65:41645)
0925 09:30:04.908278 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:04.908365 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":218,"threads_started":1}
I20260925 09:30:05.345726 26148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:05.345033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.66:47819)
0925 09:30:05.345479 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:05.345558 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":261,"threads_started":1}
I20260925 09:30:05.777072 26149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:05.776361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.67:33141)
0925 09:30:05.776798 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:05.776879 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":225,"threads_started":1}
I20260925 09:30:05.910246 26150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:05.909507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.65:40583)
0925 09:30:05.909975 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:05.910052 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":219,"threads_started":1}
I20260925 09:30:06.347242 26151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:06.346567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.66:39321)
0925 09:30:06.346977 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:06.347068 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":207,"threads_started":1}
I20260925 09:30:06.778643 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:06.777984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.67:58279)
0925 09:30:06.778402 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:06.778473 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":237,"threads_started":1}
I20260925 09:30:06.912055 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:06.911264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.65:45757)
0925 09:30:06.911740 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:06.911879 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":249,"threads_started":1}
I20260925 09:30:07.348865 26154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:07.348086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.66:44793)
0925 09:30:07.348580 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:07.348672 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":267,"threads_started":1}
I20260925 09:30:07.780680 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:07.779696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.67:47171)
0925 09:30:07.780385 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:07.780484 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":417,"threads_started":1}
I20260925 09:30:07.913686 26156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:07.912967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.65:42489)
0925 09:30:07.913426 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:07.913511 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":230,"threads_started":1}
I20260925 09:30:08.350564 26157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:08.349791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.66:45909)
0925 09:30:08.350292 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:08.350377 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":256,"threads_started":1}
I20260925 09:30:08.782382 26158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:08.781584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.67:32779)
0925 09:30:08.781995 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:08.782164 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":209,"threads_started":1}
I20260925 09:30:08.915575 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:08.914736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.65:42923)
0925 09:30:08.915177 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:08.915317 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":228,"threads_started":1}
I20260925 09:30:09.352456 26160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:09.351706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.66:44861)
0925 09:30:09.352196 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:09.352284 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":274,"threads_started":1}
I20260925 09:30:09.763450 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 25689
I20260925 09:30:09.784376 26162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:09.783476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:38571 (local address 127.10.114.67:36457)
0925 09:30:09.784083 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:09.784183 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:38571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":401,"threads_started":1}
I20260925 09:30:09.788388 26161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:09.764472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53660 (local address 127.10.114.65:46239)
0925 09:30:09.765526 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:09.765550 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:09.765589 (+ 39us) server_negotiation.cc:407] Connection header received
0925 09:30:09.767001 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:09.767009 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:09.767442 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:09.767731 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:30:09.769168 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:09.770408 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:09.771734 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:09.772434 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:09.772555 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:09.781768 (+ 9213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:30:09.781776 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:30:09.781791 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:30:09.781828 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:30:09.785190 (+ 3362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:09.785832 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:09.785838 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:09.785844 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:09.786010 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:09.786338 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:09.786341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:09.786342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:09.786843 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:30:09.787072 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:09.787683 (+ 611us) server_negotiation.cc:299] Negotiation successful
0925 09:30:09.788110 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":852,"thread_start_us":715,"threads_started":1}
I20260925 09:30:09.789749 25846 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:30:09.872188 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 25689
I20260925 09:30:09.889370 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 25854
I20260925 09:30:09.907692 26164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:09.890471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57522 (local address 127.10.114.66:38941)
0925 09:30:09.891052 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:09.891068 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:09.891107 (+ 39us) server_negotiation.cc:407] Connection header received
0925 09:30:09.892860 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:09.892871 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:09.893319 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:09.893683 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:30:09.895233 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:09.896558 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:09.897847 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:09.898551 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:09.898771 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:09.901671 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:30:09.901678 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:30:09.901692 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:30:09.901738 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:30:09.904729 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:09.905373 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:09.905380 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:09.905386 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:09.905541 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:09.905859 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:09.905862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:09.905863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:09.906342 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:30:09.906571 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:09.906979 (+ 408us) server_negotiation.cc:299] Negotiation successful
0925 09:30:09.907414 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":285,"threads_started":1}
I20260925 09:30:09.909180 25977 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:30:09.990404 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 25854
I20260925 09:30:10.007450 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 25985
I20260925 09:30:10.025267 26166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:10.008546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36124 (local address 127.10.114.67:34895)
0925 09:30:10.009122 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:10.009136 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:10.009178 (+ 42us) server_negotiation.cc:407] Connection header received
0925 09:30:10.010763 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:10.010772 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:10.011226 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:10.011522 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:30:10.012949 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:10.014126 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:10.015362 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:10.016582 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:10.016782 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:10.019203 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:30:10.019210 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:30:10.019225 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:30:10.019262 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:30:10.022265 (+ 3003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:10.022941 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:10.022947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:10.022953 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:10.023111 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:10.023466 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:10.023469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:10.023471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:10.024052 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:30:10.024273 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:10.024647 (+ 374us) server_negotiation.cc:299] Negotiation successful
0925 09:30:10.024986 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":282,"threads_started":1}
I20260925 09:30:10.026763 26108 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:30:10.108668 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 25985
2026-09-25T09:30:10Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24503 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:30:10 dist-test-slave-9pj8 krb5kdc[26170](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:30:10 dist-test-slave-9pj8 krb5kdc[26170](info): set up 2 sockets
Sep 25 09:30:10 dist-test-slave-9pj8 krb5kdc[26170](info): commencing operation
krb5kdc: starting...
W20260925 09:30:11.284456 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:30:11 dist-test-slave-9pj8 krb5kdc[26170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:30:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:30:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:30:11.523243 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:40439
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:40439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:30:11.841185 26183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:30:11.841444 26183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:30:11.841560 26183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:30:11.848562 26183 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 09:30:11.848640 26183 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 09:30:11.848716 26183 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 09:30:11.848806 26183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:30:11.848889 26183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:30:11.849002 26183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:30:11.849077 26183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:30:11.849159 26183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:30:11.858350 26183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:40439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:40439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:30:11.859619 26183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:30:11.860917 26183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:30:11.872802 26188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:11.872737 26191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:11.872869 26189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:30:11.873829 26183 server_base.cc:1034] running on GCE node
I20260925 09:30:11.874528 26183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:30:11.875908 26183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:30:11.877135 26183 hybrid_clock.cc:648] HybridClock initialized: now 1790328611877053 us; error 65 us; skew 500 ppm
I20260925 09:30:11.877490 26183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:30:11 dist-test-slave-9pj8 krb5kdc[26170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328611, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:30:11.880517 26183 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:30:11.881896 26183 webserver.cc:460] Webserver started at http://127.10.114.126:33937/ using document root <none> and password file <none>
I20260925 09:30:11.882556 26183 fs_manager.cc:359] Metadata directory not provided
I20260925 09:30:11.882681 26183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:30:11.882992 26183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:30:11.885756 26183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "a613cdc33a34404bbe16d80623c99f25"
format_stamp: "Formatted at 2026-09-25 09:30:11 on dist-test-slave-9pj8"
server_key: "3981ca57af12dd69e0328c8b4e2a56d9"
server_key_iv: "b4a130dbea1923bbda017dac5b3c516b"
server_key_version: "encryptionkey@0"
I20260925 09:30:11.886492 26183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "a613cdc33a34404bbe16d80623c99f25"
format_stamp: "Formatted at 2026-09-25 09:30:11 on dist-test-slave-9pj8"
server_key: "3981ca57af12dd69e0328c8b4e2a56d9"
server_key_iv: "b4a130dbea1923bbda017dac5b3c516b"
server_key_version: "encryptionkey@0"
I20260925 09:30:11.891263 26183 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 09:30:11.894650 26198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:11.896140 26183 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:30:11.896356 26183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "a613cdc33a34404bbe16d80623c99f25"
format_stamp: "Formatted at 2026-09-25 09:30:11 on dist-test-slave-9pj8"
server_key: "3981ca57af12dd69e0328c8b4e2a56d9"
server_key_iv: "b4a130dbea1923bbda017dac5b3c516b"
server_key_version: "encryptionkey@0"
I20260925 09:30:11.896562 26183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:30:11.914078 26183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:30:11.918011 26183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:30:11.918279 26183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:30:11.936509 26183 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:40439
I20260925 09:30:11.936537 26249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:40439 every 8 connection(s)
I20260925 09:30:11.937824 26183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:30:11.941425 26250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:30:11.943444 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 26183
I20260925 09:30:11.943671 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:30:11.944525 10697 external_mini_cluster.cc:1428] Setting key 13abe07d8538f743ca18a6a164007cf3
I20260925 09:30:11.953071 26250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25: Bootstrap starting.
I20260925 09:30:11.956067 26250 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25: Neither blocks nor log segments found. Creating new log.
I20260925 09:30:11.957029 26250 log.cc:824] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25: Log is configured to *not* fsync() on all Append() calls
Sep 25 09:30:11 dist-test-slave-9pj8 krb5kdc[26170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:30:11.960307 26250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25: No bootstrap required, opened a new log
I20260925 09:30:11.967298 26253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:11.946728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35940 (local address 127.10.114.126:40439)
0925 09:30:11.947618 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:11.947630 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:11.947677 (+ 47us) server_negotiation.cc:407] Connection header received
0925 09:30:11.949410 (+ 1733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:11.949432 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:11.949826 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:11.950221 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:30:11.951670 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:11.953000 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:11.954383 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:11.955060 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:11.955256 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:11.960248 (+ 4992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:30:11.960256 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:30:11.960269 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:30:11.960304 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:30:11.963319 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:11.964130 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:11.964142 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:11.964148 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:11.964308 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:11.964681 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:11.964686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:11.964695 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:11.965179 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:30:11.965497 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:11.965956 (+ 459us) server_negotiation.cc:299] Negotiation successful
0925 09:30:11.966349 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":369,"threads_started":1}
I20260925 09:30:11.968267 26250 raft_consensus.cc:348] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a613cdc33a34404bbe16d80623c99f25" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40439 } }
I20260925 09:30:11.968627 26250 raft_consensus.cc:374] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:30:11.968748 26250 raft_consensus.cc:729] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a613cdc33a34404bbe16d80623c99f25, State: Initialized, Role: FOLLOWER
I20260925 09:30:11.969269 26250 consensus_queue.cc:260] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a613cdc33a34404bbe16d80623c99f25" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40439 } }
I20260925 09:30:11.969525 26250 raft_consensus.cc:388] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:30:11.969668 26250 raft_consensus.cc:482] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:30:11.969841 26250 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:30:11.972039 26250 raft_consensus.cc:504] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a613cdc33a34404bbe16d80623c99f25" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40439 } }
I20260925 09:30:11.972483 26250 leader_election.cc:302] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a613cdc33a34404bbe16d80623c99f25; no voters:
I20260925 09:30:11.973029 26250 leader_election.cc:288] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:30:11.973161 26255 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:30:11.973860 26255 raft_consensus.cc:686] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 1 LEADER]: Becoming Leader. State: Replica: a613cdc33a34404bbe16d80623c99f25, State: Running, Role: LEADER
I20260925 09:30:11.974375 26255 consensus_queue.cc:237] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a613cdc33a34404bbe16d80623c99f25" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40439 } }
I20260925 09:30:11.975153 26250 sys_catalog.cc:564] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:30:11.976899 26257 sys_catalog.cc:455] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a613cdc33a34404bbe16d80623c99f25. Latest consensus state: current_term: 1 leader_uuid: "a613cdc33a34404bbe16d80623c99f25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a613cdc33a34404bbe16d80623c99f25" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40439 } } }
I20260925 09:30:11.977312 26256 sys_catalog.cc:455] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a613cdc33a34404bbe16d80623c99f25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a613cdc33a34404bbe16d80623c99f25" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40439 } } }
I20260925 09:30:11.977890 26257 sys_catalog.cc:458] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [sys.catalog]: This master's current role is: LEADER
I20260925 09:30:11.977947 26256 sys_catalog.cc:458] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [sys.catalog]: This master's current role is: LEADER
I20260925 09:30:11.978907 26262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:30:11.990936 26262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:30:11.993395 26272 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 09:30:11.999219 26262 catalog_manager.cc:1347] Generated new cluster ID: 12ef670e78cc4c03b0e96a79717fef37
I20260925 09:30:11.999372 26262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:30:12.021313 26262 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:30:12.022694 26262 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:30:12.031191 26262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25: Generated new TSK 0
I20260925 09:30:12.032222 26262 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
I20260925 09:30:12.091149 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:40439
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:30:12.410156 26276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:30:12.410392 26276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:30:12.410480 26276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:30:12.417110 26276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:30:12.417210 26276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:30:12.417312 26276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:30:12.426909 26276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:40439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:30:12.428081 26276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:30:12.429265 26276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:30:12.442203 26281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:12.442233 26284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:12.442157 26282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:30:12.442878 26276 server_base.cc:1034] running on GCE node
I20260925 09:30:12.443495 26276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:30:12.444416 26276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:30:12.445618 26276 hybrid_clock.cc:648] HybridClock initialized: now 1790328612445561 us; error 32 us; skew 500 ppm
I20260925 09:30:12.445981 26276 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:30:12 dist-test-slave-9pj8 krb5kdc[26170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328612, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:30:12.449563 26276 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:30:12.450932 26276 webserver.cc:460] Webserver started at http://127.10.114.65:39533/ using document root <none> and password file <none>
I20260925 09:30:12.451532 26276 fs_manager.cc:359] Metadata directory not provided
I20260925 09:30:12.451629 26276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:30:12.451939 26276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:30:12.454526 26276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "2a9f4e4a791044729f31d00e037e3845"
format_stamp: "Formatted at 2026-09-25 09:30:12 on dist-test-slave-9pj8"
server_key: "43fd67f0c87a53c63fa6a45f42b3f6cd"
server_key_iv: "e3d7fea6aac2ea4ea3ad896acd4c8c3c"
server_key_version: "encryptionkey@0"
I20260925 09:30:12.455200 26276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "2a9f4e4a791044729f31d00e037e3845"
format_stamp: "Formatted at 2026-09-25 09:30:12 on dist-test-slave-9pj8"
server_key: "43fd67f0c87a53c63fa6a45f42b3f6cd"
server_key_iv: "e3d7fea6aac2ea4ea3ad896acd4c8c3c"
server_key_version: "encryptionkey@0"
I20260925 09:30:12.460088 26276 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260925 09:30:12.463331 26291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:12.464702 26276 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:30:12.464854 26276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "2a9f4e4a791044729f31d00e037e3845"
format_stamp: "Formatted at 2026-09-25 09:30:12 on dist-test-slave-9pj8"
server_key: "43fd67f0c87a53c63fa6a45f42b3f6cd"
server_key_iv: "e3d7fea6aac2ea4ea3ad896acd4c8c3c"
server_key_version: "encryptionkey@0"
I20260925 09:30:12.465005 26276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:30:12.482743 26276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:30:12.486585 26276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:30:12.486809 26276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:30:12.487646 26276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:30:12.488946 26276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:30:12.489032 26276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:12.489114 26276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:30:12.489157 26276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:12.527863 26276 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:38917
I20260925 09:30:12.528016 26403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:38917 every 8 connection(s)
I20260925 09:30:12.529328 26276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:30:12.531675 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 26276
I20260925 09:30:12.531917 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:30:12.532630 10697 external_mini_cluster.cc:1428] Setting key 69d74ddae25079ec158c8e756899dce7
Sep 25 09:30:12 dist-test-slave-9pj8 krb5kdc[26170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328612, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:30:12.551914 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:12.531951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:41739 (local address 127.10.114.126:40439)
0925 09:30:12.532413 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:12.532419 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:12.535562 (+ 3143us) server_negotiation.cc:407] Connection header received
0925 09:30:12.535615 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:12.535618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:12.535713 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:12.535877 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:30:12.537894 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:12.538591 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:12.541114 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:12.541514 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:12.541609 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:12.545236 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:30:12.545244 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:30:12.545252 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:30:12.545296 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:30:12.548868 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:12.549482 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:12.549490 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:12.549496 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:12.549633 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:12.550017 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:12.550025 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:12.550031 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:12.550399 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:30:12.550565 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:12.551180 (+ 615us) server_negotiation.cc:299] Negotiation successful
0925 09:30:12.551423 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":214,"threads_started":1}
I20260925 09:30:12.552465 26407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:12.532552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.65:41739)
0925 09:30:12.533419 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:12.533478 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 09:30:12.535293 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:30:12.536125 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:30:12.536138 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:30:12.536943 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:30:12.537712 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:12.537740 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:12.538779 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:12.538787 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:12.540974 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:12.540992 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:12.541734 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:12.541743 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:12.542058 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:12.542767 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:30:12.542799 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:30:12.545026 (+ 2227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:30:12.548992 (+ 3966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:30:12.548998 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:30:12.549009 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:30:12.549360 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:30:12.549735 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:30:12.549738 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:30:12.549740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:30:12.549899 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:30:12.550536 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:30:12.550542 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:30:12.550811 (+ 269us) client_negotiation.cc:769] Sending connection context
0925 09:30:12.551026 (+ 215us) client_negotiation.cc:240] Negotiation successful
0925 09:30:12.551361 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":349,"threads_started":1}
I20260925 09:30:12.553996 26404 heartbeater.cc:347] Connected to a master server at 127.10.114.126:40439
I20260925 09:30:12.554314 26404 heartbeater.cc:464] Registering TS with master...
I20260925 09:30:12.555234 26404 heartbeater.cc:511] Master 127.10.114.126:40439 requested a full tablet report, sending...
I20260925 09:30:12.557581 26215 ts_manager.cc:194] Registered new tserver with Master: 2a9f4e4a791044729f31d00e037e3845 (127.10.114.65:38917)
WARNING: no policy specified for kudu/127.10.114.66@KRBTEST.COM; defaulting to no policy
I20260925 09:30:12.559074 26215 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:41739
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
I20260925 09:30:12.584815 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:40439
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:30:12.895128 26410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:30:12.895402 26410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:30:12.895572 26410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:30:12.903039 26410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:30:12.903142 26410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:30:12.903379 26410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:30:12.912645 26410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:40439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:30:12.913843 26410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:30:12.915156 26410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:30:12.928033 26415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:12.928117 26416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:12.928155 26418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:30:12.929056 26410 server_base.cc:1034] running on GCE node
I20260925 09:30:12.929631 26410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:30:12.930529 26410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:30:12.931811 26410 hybrid_clock.cc:648] HybridClock initialized: now 1790328612931741 us; error 62 us; skew 500 ppm
I20260925 09:30:12.932121 26410 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:30:12 dist-test-slave-9pj8 krb5kdc[26170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328612, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:30:12.935081 26410 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:30:12.936576 26410 webserver.cc:460] Webserver started at http://127.10.114.66:37829/ using document root <none> and password file <none>
I20260925 09:30:12.937227 26410 fs_manager.cc:359] Metadata directory not provided
I20260925 09:30:12.937361 26410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:30:12.937673 26410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:30:12.940497 26410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "ab1d10dc32c3439f8dab35ca8c44afe2"
format_stamp: "Formatted at 2026-09-25 09:30:12 on dist-test-slave-9pj8"
server_key: "0e992f59ce9eb8f8da441b4bda81ecd3"
server_key_iv: "40557a36f9ea460098fdd6125cefef0e"
server_key_version: "encryptionkey@0"
I20260925 09:30:12.941241 26410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "ab1d10dc32c3439f8dab35ca8c44afe2"
format_stamp: "Formatted at 2026-09-25 09:30:12 on dist-test-slave-9pj8"
server_key: "0e992f59ce9eb8f8da441b4bda81ecd3"
server_key_iv: "40557a36f9ea460098fdd6125cefef0e"
server_key_version: "encryptionkey@0"
I20260925 09:30:12.946069 26410 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:30:12.949453 26425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:12.950701 26410 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 09:30:12.950812 26410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "ab1d10dc32c3439f8dab35ca8c44afe2"
format_stamp: "Formatted at 2026-09-25 09:30:12 on dist-test-slave-9pj8"
server_key: "0e992f59ce9eb8f8da441b4bda81ecd3"
server_key_iv: "40557a36f9ea460098fdd6125cefef0e"
server_key_version: "encryptionkey@0"
I20260925 09:30:12.950965 26410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:30:12.981827 26410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:30:12.985492 26410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:30:12.985723 26410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:30:12.986555 26410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:30:12.987722 26410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:30:12.987836 26410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:12.987946 26410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:30:12.987989 26410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:13.020848 26410 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:33579
I20260925 09:30:13.020890 26537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:33579 every 8 connection(s)
I20260925 09:30:13.022087 26410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:30:13.024212 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 26410
I20260925 09:30:13.024400 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:30:13.024981 10697 external_mini_cluster.cc:1428] Setting key 24b30573e4b492d2f06e3161f0abc6f9
Sep 25 09:30:13 dist-test-slave-9pj8 krb5kdc[26170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328612, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:30:13.044979 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:13.024868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:60939 (local address 127.10.114.126:40439)
0925 09:30:13.025075 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:13.025081 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:13.026351 (+ 1270us) server_negotiation.cc:407] Connection header received
0925 09:30:13.029049 (+ 2698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:13.029053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:13.029115 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:13.029214 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:30:13.031215 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.032153 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.034594 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.034995 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.035082 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:13.039310 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:30:13.039315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:30:13.039318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:30:13.039348 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:30:13.041981 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:13.042846 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:13.042850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:13.042852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:13.042969 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:13.043425 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:13.043427 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:13.043429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:13.043663 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:30:13.043766 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:13.044584 (+ 818us) server_negotiation.cc:299] Negotiation successful
0925 09:30:13.044742 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 09:30:13.046494 26540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:13.025334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.66:60939)
0925 09:30:13.026186 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:13.026237 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 09:30:13.028757 (+ 2520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:30:13.029479 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:30:13.029492 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:30:13.030241 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:30:13.031024 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:13.031050 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.032324 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.032332 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:13.034435 (+ 2103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:13.034452 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.035217 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.035226 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:13.035536 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:13.036277 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:30:13.036305 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:30:13.039072 (+ 2767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:30:13.042113 (+ 3041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:30:13.042124 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:30:13.042142 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:30:13.042741 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:30:13.043080 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:30:13.043088 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:30:13.043093 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:30:13.043325 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:30:13.043807 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:30:13.043817 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:30:13.044241 (+ 424us) client_negotiation.cc:769] Sending connection context
0925 09:30:13.044531 (+ 290us) client_negotiation.cc:240] Negotiation successful
0925 09:30:13.045045 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":374,"threads_started":1}
I20260925 09:30:13.047916 26538 heartbeater.cc:347] Connected to a master server at 127.10.114.126:40439
I20260925 09:30:13.048254 26538 heartbeater.cc:464] Registering TS with master...
I20260925 09:30:13.049124 26538 heartbeater.cc:511] Master 127.10.114.126:40439 requested a full tablet report, sending...
I20260925 09:30:13.050786 26215 ts_manager.cc:194] Registered new tserver with Master: ab1d10dc32c3439f8dab35ca8c44afe2 (127.10.114.66:33579)
WARNING: no policy specified for kudu/127.10.114.67@KRBTEST.COM; defaulting to no policy
I20260925 09:30:13.051707 26215 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:60939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
I20260925 09:30:13.079349 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:40439
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:30:13.397040 26543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:30:13.397295 26543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:30:13.397465 26543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:30:13.404618 26543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:30:13.404709 26543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:30:13.404920 26543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:30:13.414703 26543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:40439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:30:13.415930 26543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:30:13.417137 26543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:30:13.429731 26549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:13.429802 26548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:13.430353 26551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:30:13.430459 26543 server_base.cc:1034] running on GCE node
I20260925 09:30:13.431059 26543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:30:13.432022 26543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:30:13.433229 26543 hybrid_clock.cc:648] HybridClock initialized: now 1790328613433173 us; error 51 us; skew 500 ppm
I20260925 09:30:13.433481 26543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:30:13 dist-test-slave-9pj8 krb5kdc[26170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328613, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:30:13.436463 26543 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:30:13.437870 26543 webserver.cc:460] Webserver started at http://127.10.114.67:39227/ using document root <none> and password file <none>
I20260925 09:30:13.438542 26543 fs_manager.cc:359] Metadata directory not provided
I20260925 09:30:13.438643 26543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:30:13.438891 26543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:30:13.441737 26543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "d8fea1757f77441b9115d8680c4f085d"
format_stamp: "Formatted at 2026-09-25 09:30:13 on dist-test-slave-9pj8"
server_key: "d3e50694b23a184bbf5096de9ddb0d94"
server_key_iv: "a6fd0ef9722b278a3be7b529789dac4f"
server_key_version: "encryptionkey@0"
I20260925 09:30:13.442420 26543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "d8fea1757f77441b9115d8680c4f085d"
format_stamp: "Formatted at 2026-09-25 09:30:13 on dist-test-slave-9pj8"
server_key: "d3e50694b23a184bbf5096de9ddb0d94"
server_key_iv: "a6fd0ef9722b278a3be7b529789dac4f"
server_key_version: "encryptionkey@0"
I20260925 09:30:13.447346 26543 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260925 09:30:13.450644 26558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:13.452059 26543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:30:13.452297 26543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "d8fea1757f77441b9115d8680c4f085d"
format_stamp: "Formatted at 2026-09-25 09:30:13 on dist-test-slave-9pj8"
server_key: "d3e50694b23a184bbf5096de9ddb0d94"
server_key_iv: "a6fd0ef9722b278a3be7b529789dac4f"
server_key_version: "encryptionkey@0"
I20260925 09:30:13.452524 26543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:30:13.467694 26543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:30:13.472366 26543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:30:13.472710 26543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:30:13.473906 26543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:30:13.475749 26543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:30:13.476042 26543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260925 09:30:13.476179 26543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:30:13.476235 26543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:13.510496 26543 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:40219
I20260925 09:30:13.510547 26670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:40219 every 8 connection(s)
I20260925 09:30:13.511900 26543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:30:13.518778 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 26543
I20260925 09:30:13.519016 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:30:13.519645 10697 external_mini_cluster.cc:1428] Setting key f9cf2cbe98103261957abcf4b7f127be
Sep 25 09:30:13 dist-test-slave-9pj8 krb5kdc[26170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328613, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:30:13.536213 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:13.514573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:56399 (local address 127.10.114.126:40439)
0925 09:30:13.514823 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:13.514832 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:13.516389 (+ 1557us) server_negotiation.cc:407] Connection header received
0925 09:30:13.519599 (+ 3210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:13.519607 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:13.519711 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:13.519826 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:30:13.522388 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.523686 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.525483 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.525882 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.525985 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:13.529497 (+ 3512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:30:13.529503 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:30:13.529508 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:30:13.529546 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:30:13.532914 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:13.533659 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:13.533662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:13.533665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:13.533836 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:13.534328 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:13.534335 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:13.534338 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:13.534669 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:30:13.534815 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:13.535637 (+ 822us) server_negotiation.cc:299] Negotiation successful
0925 09:30:13.535876 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 09:30:13.537070 26673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:13.515137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.67:56399)
0925 09:30:13.516176 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:13.516239 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0925 09:30:13.519163 (+ 2924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:30:13.520430 (+ 1267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:30:13.520446 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:30:13.521310 (+ 864us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:30:13.522203 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:13.522226 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.523939 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.523945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:13.525336 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:13.525349 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.526061 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.526065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:13.526256 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:13.526824 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:30:13.526845 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:30:13.529300 (+ 2455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:30:13.533045 (+ 3745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:30:13.533052 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:30:13.533063 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:30:13.533521 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:30:13.533968 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:30:13.533974 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:30:13.533977 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:30:13.534205 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:30:13.534830 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:30:13.534838 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:30:13.535187 (+ 349us) client_negotiation.cc:769] Sending connection context
0925 09:30:13.535518 (+ 331us) client_negotiation.cc:240] Negotiation successful
0925 09:30:13.536045 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":374,"threads_started":1}
I20260925 09:30:13.538481 26671 heartbeater.cc:347] Connected to a master server at 127.10.114.126:40439
I20260925 09:30:13.538890 26671 heartbeater.cc:464] Registering TS with master...
I20260925 09:30:13.539593 26671 heartbeater.cc:511] Master 127.10.114.126:40439 requested a full tablet report, sending...
I20260925 09:30:13.540983 26215 ts_manager.cc:194] Registered new tserver with Master: d8fea1757f77441b9115d8680c4f085d (127.10.114.67:40219)
I20260925 09:30:13.541698 26215 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:56399
I20260925 09:30:13.547399 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:30:13.562306 26404 heartbeater.cc:503] Master 127.10.114.126:40439 was elected leader, sending a full tablet report...
I20260925 09:30:13.564800 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:13.551696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51886 (local address 127.10.114.126:40439)
0925 09:30:13.551904 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:13.551912 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:13.552319 (+ 407us) server_negotiation.cc:407] Connection header received
0925 09:30:13.552502 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:13.552507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:13.552574 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:13.552668 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:30:13.554117 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.554797 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.556443 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.556898 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.557030 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:13.559209 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:30:13.559214 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:30:13.559217 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:30:13.559247 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:30:13.562032 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:13.562727 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:13.562731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:13.562734 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:13.562918 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:13.563353 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:13.563356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:13.563358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:13.563659 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:30:13.564004 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:13.564286 (+ 282us) server_negotiation.cc:299] Negotiation successful
0925 09:30:13.564434 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 09:30:13.568480 26215 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51886:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:30:13.575192 26215 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:30:13.598242 26683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:13.586536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:33579 (local address 127.0.0.1:57586)
0925 09:30:13.588155 (+ 1619us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:13.588202 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 09:30:13.588367 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:30:13.590056 (+ 1689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:30:13.590064 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:30:13.590101 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:30:13.590506 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:13.590518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.592423 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.592427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:13.594901 (+ 2474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:13.594916 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.597144 (+ 2228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.597151 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:13.597346 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:13.597421 (+ 75us) client_negotiation.cc:769] Sending connection context
0925 09:30:13.597554 (+ 133us) client_negotiation.cc:240] Negotiation successful
0925 09:30:13.597745 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1376,"mutex_wait_us":96,"thread_start_us":1234,"threads_started":1}
I20260925 09:30:13.599541 26684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:13.586668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57586 (local address 127.10.114.66:33579)
0925 09:30:13.589284 (+ 2616us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:13.589292 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:13.589314 (+ 22us) server_negotiation.cc:407] Connection header received
0925 09:30:13.589410 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:13.589415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:13.589587 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:13.589787 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:30:13.590692 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.592265 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.595097 (+ 2832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.596675 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.596799 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:13.596997 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:13.599030 (+ 2033us) server_negotiation.cc:299] Negotiation successful
0925 09:30:13.599207 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2479,"thread_start_us":2348,"threads_started":1}
I20260925 09:30:13.600575 26685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:13.586754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:38917 (local address 127.0.0.1:59948)
0925 09:30:13.588850 (+ 2096us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:13.588876 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 09:30:13.589004 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:30:13.590965 (+ 1961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:30:13.590969 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:30:13.590992 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:30:13.591392 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:13.591404 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.594118 (+ 2714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.594125 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:13.596389 (+ 2264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:13.596406 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.599630 (+ 3224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.599636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:13.599879 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:13.599942 (+ 63us) client_negotiation.cc:769] Sending connection context
0925 09:30:13.600085 (+ 143us) client_negotiation.cc:240] Negotiation successful
0925 09:30:13.600220 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1988,"mutex_wait_us":4,"thread_start_us":2060,"threads_started":1}
I20260925 09:30:13.600736 26686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:13.587498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59948 (local address 127.10.114.65:38917)
0925 09:30:13.588876 (+ 1378us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:13.588885 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:13.589130 (+ 245us) server_negotiation.cc:407] Connection header received
0925 09:30:13.589229 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:13.589237 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:13.589478 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:13.590707 (+ 1229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:30:13.591584 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.593064 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.596571 (+ 3507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.599127 (+ 2556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.599250 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:13.599485 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:13.600220 (+ 735us) server_negotiation.cc:299] Negotiation successful
0925 09:30:13.600411 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1204,"thread_start_us":1032,"threads_started":1}
I20260925 09:30:13.603860 26473 tablet_service.cc:1467] Processing CreateTablet for tablet 28524542955a4c3dab832a6bc738f4cd (DEFAULT_TABLE table=test-table [id=5a23b3872d994cb7864268ce63a83d4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:30:13.604378 26339 tablet_service.cc:1467] Processing CreateTablet for tablet 28524542955a4c3dab832a6bc738f4cd (DEFAULT_TABLE table=test-table [id=5a23b3872d994cb7864268ce63a83d4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:30:13.605309 26473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28524542955a4c3dab832a6bc738f4cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:30:13.605646 26339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28524542955a4c3dab832a6bc738f4cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:30:13.608410 26683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:13.589209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:40219 (local address 127.0.0.1:44662)
0925 09:30:13.598489 (+ 9280us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:13.598509 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:30:13.598605 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:30:13.601806 (+ 3201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:30:13.601811 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:30:13.601842 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:30:13.602037 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:13.602051 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.604100 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.604105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:13.605770 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:13.605785 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.607583 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.607591 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:13.607815 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:13.607871 (+ 56us) client_negotiation.cc:769] Sending connection context
0925 09:30:13.607972 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 09:30:13.608108 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9189,"thread_start_us":10667,"threads_started":1}
I20260925 09:30:13.608938 26688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:13.589764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44662 (local address 127.10.114.67:40219)
0925 09:30:13.600592 (+ 10828us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:13.600602 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:13.601219 (+ 617us) server_negotiation.cc:407] Connection header received
0925 09:30:13.601347 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:13.601366 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:13.601619 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:13.601820 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:30:13.602263 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.603922 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.605933 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.607401 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.607554 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:13.607838 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:13.608038 (+ 200us) server_negotiation.cc:299] Negotiation successful
0925 09:30:13.608200 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10645,"thread_start_us":11361,"threads_started":1}
I20260925 09:30:13.612757 26606 tablet_service.cc:1467] Processing CreateTablet for tablet 28524542955a4c3dab832a6bc738f4cd (DEFAULT_TABLE table=test-table [id=5a23b3872d994cb7864268ce63a83d4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:30:13.614141 26606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28524542955a4c3dab832a6bc738f4cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:30:13.617952 26689 tablet_bootstrap.cc:492] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845: Bootstrap starting.
I20260925 09:30:13.619977 26690 tablet_bootstrap.cc:492] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2: Bootstrap starting.
I20260925 09:30:13.621944 26689 tablet_bootstrap.cc:654] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845: Neither blocks nor log segments found. Creating new log.
I20260925 09:30:13.623258 26689 log.cc:824] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845: Log is configured to *not* fsync() on all Append() calls
I20260925 09:30:13.623956 26690 tablet_bootstrap.cc:654] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2: Neither blocks nor log segments found. Creating new log.
I20260925 09:30:13.625473 26690 log.cc:824] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2: Log is configured to *not* fsync() on all Append() calls
I20260925 09:30:13.627048 26693 tablet_bootstrap.cc:492] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d: Bootstrap starting.
I20260925 09:30:13.628175 26689 tablet_bootstrap.cc:492] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845: No bootstrap required, opened a new log
I20260925 09:30:13.628454 26689 ts_tablet_manager.cc:1397] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260925 09:30:13.629107 26690 tablet_bootstrap.cc:492] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2: No bootstrap required, opened a new log
I20260925 09:30:13.629393 26690 ts_tablet_manager.cc:1397] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260925 09:30:13.631074 26693 tablet_bootstrap.cc:654] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d: Neither blocks nor log segments found. Creating new log.
I20260925 09:30:13.632328 26693 log.cc:824] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d: Log is configured to *not* fsync() on all Append() calls
I20260925 09:30:13.635284 26693 tablet_bootstrap.cc:492] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d: No bootstrap required, opened a new log
I20260925 09:30:13.635532 26693 ts_tablet_manager.cc:1397] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260925 09:30:13.637080 26690 raft_consensus.cc:348] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } } peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } }
I20260925 09:30:13.637439 26690 raft_consensus.cc:374] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:30:13.637583 26690 raft_consensus.cc:729] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab1d10dc32c3439f8dab35ca8c44afe2, State: Initialized, Role: FOLLOWER
I20260925 09:30:13.638224 26690 consensus_queue.cc:260] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } } peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } }
I20260925 09:30:13.639365 26689 raft_consensus.cc:348] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } } peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } }
I20260925 09:30:13.639851 26689 raft_consensus.cc:374] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:30:13.639981 26689 raft_consensus.cc:729] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9f4e4a791044729f31d00e037e3845, State: Initialized, Role: FOLLOWER
I20260925 09:30:13.640697 26538 heartbeater.cc:503] Master 127.10.114.126:40439 was elected leader, sending a full tablet report...
I20260925 09:30:13.640671 26689 consensus_queue.cc:260] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } } peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } }
I20260925 09:30:13.641306 26690 ts_tablet_manager.cc:1428] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2: Time spent starting tablet: real 0.012s user 0.006s sys 0.006s
I20260925 09:30:13.642433 26689 ts_tablet_manager.cc:1428] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260925 09:30:13.647094 26693 raft_consensus.cc:348] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } } peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } }
I20260925 09:30:13.647548 26693 raft_consensus.cc:374] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:30:13.647683 26693 raft_consensus.cc:729] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8fea1757f77441b9115d8680c4f085d, State: Initialized, Role: FOLLOWER
I20260925 09:30:13.648236 26693 consensus_queue.cc:260] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } } peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } }
I20260925 09:30:13.649294 26671 heartbeater.cc:503] Master 127.10.114.126:40439 was elected leader, sending a full tablet report...
I20260925 09:30:13.649892 26693 ts_tablet_manager.cc:1428] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260925 09:30:13.763558 26672 tablet.cc:2369] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:30:13.774281 26539 tablet.cc:2369] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:30:13.781555 26405 tablet.cc:2369] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:30:13.963591 26695 raft_consensus.cc:482] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:30:13.963886 26695 raft_consensus.cc:504] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } } peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } }
I20260925 09:30:13.965425 26695 leader_election.cc:288] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9f4e4a791044729f31d00e037e3845 (127.10.114.65:38917), d8fea1757f77441b9115d8680c4f085d (127.10.114.67:40219)
I20260925 09:30:13.974329 26686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:13.965855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:41059 (local address 127.10.114.65:38917)
0925 09:30:13.966138 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:13.966150 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:13.966654 (+ 504us) server_negotiation.cc:407] Connection header received
0925 09:30:13.966892 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:13.966902 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:13.967028 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:13.967136 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:30:13.967832 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.968975 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.971685 (+ 2710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.973053 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.973239 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:13.973324 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:13.973892 (+ 568us) server_negotiation.cc:299] Negotiation successful
0925 09:30:13.974029 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260925 09:30:13.974341 26688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:13.966858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:52315 (local address 127.10.114.67:40219)
0925 09:30:13.967074 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:13.967082 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:13.967446 (+ 364us) server_negotiation.cc:407] Connection header received
0925 09:30:13.967616 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:13.967620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:13.967683 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:13.967886 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:30:13.968403 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.969424 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.971732 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.973091 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.973262 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:13.973334 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:13.973892 (+ 558us) server_negotiation.cc:299] Negotiation successful
0925 09:30:13.974029 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260925 09:30:13.974560 26698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:13.965827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:38917 (local address 127.10.114.66:41059)
0925 09:30:13.966507 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:13.966523 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:30:13.966712 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:30:13.967184 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:30:13.967189 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:30:13.967215 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:30:13.967660 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:13.967673 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.969175 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.969179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:13.971444 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:13.971471 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.973377 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.973384 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:13.973635 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:13.973745 (+ 110us) client_negotiation.cc:769] Sending connection context
0925 09:30:13.973844 (+ 99us) client_negotiation.cc:240] Negotiation successful
0925 09:30:13.974048 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"mutex_wait_us":142,"thread_start_us":414,"threads_started":1}
I20260925 09:30:13.974581 26699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:13.966645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:40219 (local address 127.10.114.66:52315)
0925 09:30:13.967331 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:13.967350 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:30:13.967474 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:30:13.967883 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:30:13.967886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:30:13.967905 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:30:13.968252 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:13.968260 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.969550 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.969555 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:13.971576 (+ 2021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:13.971587 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:13.973391 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:13.973397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:13.973640 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:13.973741 (+ 101us) client_negotiation.cc:769] Sending connection context
0925 09:30:13.973844 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 09:30:13.974048 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"mutex_wait_us":114,"thread_start_us":331,"threads_started":1}
I20260925 09:30:13.975723 26626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28524542955a4c3dab832a6bc738f4cd" candidate_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8fea1757f77441b9115d8680c4f085d" is_pre_election: true
I20260925 09:30:13.975939 26359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28524542955a4c3dab832a6bc738f4cd" candidate_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9f4e4a791044729f31d00e037e3845" is_pre_election: true
I20260925 09:30:13.976243 26626 raft_consensus.cc:2454] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab1d10dc32c3439f8dab35ca8c44afe2 in term 0.
I20260925 09:30:13.976533 26359 raft_consensus.cc:2454] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab1d10dc32c3439f8dab35ca8c44afe2 in term 0.
I20260925 09:30:13.976923 26426 leader_election.cc:302] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab1d10dc32c3439f8dab35ca8c44afe2, d8fea1757f77441b9115d8680c4f085d; no voters:
I20260925 09:30:13.977385 26695 raft_consensus.cc:2781] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:30:13.977576 26695 raft_consensus.cc:482] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:30:13.977721 26695 raft_consensus.cc:3037] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:30:13.979986 26695 raft_consensus.cc:504] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } } peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } }
I20260925 09:30:13.980662 26695 leader_election.cc:288] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [CANDIDATE]: Term 1 election: Requested vote from peers 2a9f4e4a791044729f31d00e037e3845 (127.10.114.65:38917), d8fea1757f77441b9115d8680c4f085d (127.10.114.67:40219)
I20260925 09:30:13.981110 26359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28524542955a4c3dab832a6bc738f4cd" candidate_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9f4e4a791044729f31d00e037e3845"
I20260925 09:30:13.981207 26626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28524542955a4c3dab832a6bc738f4cd" candidate_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8fea1757f77441b9115d8680c4f085d"
I20260925 09:30:13.981313 26359 raft_consensus.cc:3037] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:30:13.981407 26626 raft_consensus.cc:3037] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:30:13.984375 26626 raft_consensus.cc:2454] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab1d10dc32c3439f8dab35ca8c44afe2 in term 1.
I20260925 09:30:13.984375 26359 raft_consensus.cc:2454] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab1d10dc32c3439f8dab35ca8c44afe2 in term 1.
I20260925 09:30:13.984735 26426 leader_election.cc:302] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9f4e4a791044729f31d00e037e3845, ab1d10dc32c3439f8dab35ca8c44afe2; no voters:
I20260925 09:30:13.984994 26695 raft_consensus.cc:2781] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:30:13.985448 26695 raft_consensus.cc:686] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [term 1 LEADER]: Becoming Leader. State: Replica: ab1d10dc32c3439f8dab35ca8c44afe2, State: Running, Role: LEADER
I20260925 09:30:13.986032 26695 consensus_queue.cc:237] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } } peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } }
I20260925 09:30:13.990455 26215 catalog_manager.cc:5495] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 reported cstate change: term changed from 0 to 1, leader changed from <none> to ab1d10dc32c3439f8dab35ca8c44afe2 (127.10.114.66). New cstate: current_term: 1 leader_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } health_report { overall_health: UNKNOWN } } }
I20260925 09:30:14.049574 26684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:14.043933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57602 (local address 127.10.114.66:33579)
0925 09:30:14.044172 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:14.044179 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:14.044324 (+ 145us) server_negotiation.cc:407] Connection header received
0925 09:30:14.044583 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:14.044587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:14.044681 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:14.044834 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:30:14.045195 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:14.045932 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:14.047416 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:14.047911 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:14.048048 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:14.048574 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:30:14.049040 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:30:14.049189 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:14.049260 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 09:30:14.049332 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260925 09:30:14.057883 26626 raft_consensus.cc:1261] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d [term 1 FOLLOWER]: Refusing update from remote peer ab1d10dc32c3439f8dab35ca8c44afe2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:30:14.057880 26359 raft_consensus.cc:1261] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845 [term 1 FOLLOWER]: Refusing update from remote peer ab1d10dc32c3439f8dab35ca8c44afe2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:30:14.058861 26700 consensus_queue.cc:1035] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:30:14.059237 26695 consensus_queue.cc:1035] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:30:14.083937 26214 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:30:14.084229 26214 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:30:14.087572 26214 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25: Sending DeleteTablet for 3 replicas of tablet 28524542955a4c3dab832a6bc738f4cd
I20260925 09:30:14.088713 26339 tablet_service.cc:1514] Processing DeleteTablet for tablet 28524542955a4c3dab832a6bc738f4cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:30:14 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:59948
I20260925 09:30:14.088698 26473 tablet_service.cc:1514] Processing DeleteTablet for tablet 28524542955a4c3dab832a6bc738f4cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:30:14 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:57586
I20260925 09:30:14.088878 26606 tablet_service.cc:1514] Processing DeleteTablet for tablet 28524542955a4c3dab832a6bc738f4cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:30:14 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:44662
I20260925 09:30:14.090075 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 26276
I20260925 09:30:14.117093 26715 tablet_replica.cc:330] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d: stopping tablet replica
I20260925 09:30:14.117672 26715 raft_consensus.cc:2229] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:30:14.118331 26715 raft_consensus.cc:2258] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:30:14.124610 26715 ts_tablet_manager.cc:1905] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:30:14.133853 26715 ts_tablet_manager.cc:1918] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:30:14.136319 26714 tablet_replica.cc:330] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2: stopping tablet replica
I20260925 09:30:14.136765 26714 raft_consensus.cc:2229] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:30:14.137454 26714 raft_consensus.cc:2258] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:30:14.138203 26713 tablet_replica.cc:330] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845: stopping tablet replica
I20260925 09:30:14.138612 26713 raft_consensus.cc:2229] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:30:14.139139 26713 raft_consensus.cc:2258] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:30:14.139187 26714 ts_tablet_manager.cc:1905] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 25 09:30:14 dist-test-slave-9pj8 krb5kdc[26170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:30:14.142189 26713 ts_tablet_manager.cc:1905] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:30:14.146310 26715 log.cc:1196] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/28524542955a4c3dab832a6bc738f4cd
I20260925 09:30:14.146991 26715 ts_tablet_manager.cc:1939] T 28524542955a4c3dab832a6bc738f4cd P d8fea1757f77441b9115d8680c4f085d: Deleting consensus metadata
I20260925 09:30:14.147044 26714 ts_tablet_manager.cc:1918] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:30:14.147226 26714 log.cc:1196] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/28524542955a4c3dab832a6bc738f4cd
I20260925 09:30:14.147833 26714 ts_tablet_manager.cc:1939] T 28524542955a4c3dab832a6bc738f4cd P ab1d10dc32c3439f8dab35ca8c44afe2: Deleting consensus metadata
I20260925 09:30:14.149878 26201 catalog_manager.cc:4841] TS ab1d10dc32c3439f8dab35ca8c44afe2 (127.10.114.66:33579): tablet 28524542955a4c3dab832a6bc738f4cd (table test-table [id=5a23b3872d994cb7864268ce63a83d4a]) successfully deleted
I20260925 09:30:14.150959 26199 catalog_manager.cc:4841] TS d8fea1757f77441b9115d8680c4f085d (127.10.114.67:40219): tablet 28524542955a4c3dab832a6bc738f4cd (table test-table [id=5a23b3872d994cb7864268ce63a83d4a]) successfully deleted
I20260925 09:30:14.152245 26686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:14.091409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59954 (local address 127.10.114.65:38917)
0925 09:30:14.091593 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:14.091600 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:14.113788 (+ 22188us) server_negotiation.cc:407] Connection header received
0925 09:30:14.114283 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:14.114289 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:14.114387 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:14.114465 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:30:14.118837 (+ 4372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:14.120093 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:14.126613 (+ 6520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:14.127312 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:14.127419 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:14.143554 (+ 16135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:30:14.143569 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:30:14.143589 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:30:14.143639 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:30:14.147944 (+ 4305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:14.149341 (+ 1397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:14.149351 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:14.149355 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:14.149557 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:14.150319 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:14.150325 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:14.150329 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:14.150859 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:30:14.151285 (+ 426us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:14.151528 (+ 243us) server_negotiation.cc:299] Negotiation successful
0925 09:30:14.151809 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 09:30:14.152946 26399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:30:14.239919 26713 ts_tablet_manager.cc:1918] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:30:14.240199 26713 log.cc:1196] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/28524542955a4c3dab832a6bc738f4cd
I20260925 09:30:14.240542 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 26276
I20260925 09:30:14.257905 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 26410
W20260925 09:30:14.258389 26199 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.114.65:38917 (error 108)
I20260925 09:30:14.262501 26719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:14.261705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:38917 (local address 127.0.0.1:59968)
0925 09:30:14.262174 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:14.262297 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":213,"threads_started":1}
W20260925 09:30:14.262854 26199 catalog_manager.cc:4584] TS 2a9f4e4a791044729f31d00e037e3845 (127.10.114.65:38917): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 28524542955a4c3dab832a6bc738f4cd: Network error: Client connection negotiation failed: client connection to 127.10.114.65:38917: connect: Connection refused (error 111)
Sep 25 09:30:14 dist-test-slave-9pj8 krb5kdc[26170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.66@KRBTEST.COM
I20260925 09:30:14.274418 26684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:14.259098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57610 (local address 127.10.114.66:33579)
0925 09:30:14.259307 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:14.259313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:14.259331 (+ 18us) server_negotiation.cc:407] Connection header received
0925 09:30:14.259398 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:14.259403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:14.259508 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:14.259612 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:30:14.261480 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:14.262555 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:14.264076 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:14.264504 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:14.264628 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:14.268623 (+ 3995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:30:14.268635 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:30:14.268650 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:30:14.268690 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:30:14.271626 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:14.272443 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:14.272449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:14.272451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:14.272582 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:14.273015 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:14.273018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:14.273020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:14.273348 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:30:14.273685 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:14.273891 (+ 206us) server_negotiation.cc:299] Negotiation successful
0925 09:30:14.274108 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260925 09:30:14.275080 26533 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:30:14.316639 26719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:14.316193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:38917 (local address 127.0.0.1:59972)
0925 09:30:14.316391 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:14.316487 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260925 09:30:14.317662 26719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:14.317274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:38917 (local address 127.0.0.1:59982)
0925 09:30:14.317419 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:14.317501 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 09:30:14.378187 26719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:14.377660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:38917 (local address 127.0.0.1:59988)
0925 09:30:14.377907 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:14.378015 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260925 09:30:14.379289 26719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:14.378902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:38917 (local address 127.0.0.1:59994)
0925 09:30:14.379050 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:14.379138 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260925 09:30:14.386804 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 26410
I20260925 09:30:14.406778 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 26543
Sep 25 09:30:14 dist-test-slave-9pj8 krb5kdc[26170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.67@KRBTEST.COM
I20260925 09:30:14.422051 26688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:14.407603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44676 (local address 127.10.114.67:40219)
0925 09:30:14.407847 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:14.407852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:14.407873 (+ 21us) server_negotiation.cc:407] Connection header received
0925 09:30:14.407954 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:14.407959 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:14.408036 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:14.408131 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:30:14.409584 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:14.410310 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:14.411660 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:14.412136 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:14.412235 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:14.416651 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:30:14.416664 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:30:14.416681 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:30:14.416714 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:30:14.419343 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:14.420220 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:14.420227 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:14.420229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:14.420355 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:14.420674 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:14.420677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:14.420679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:14.421030 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:30:14.421223 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:14.421532 (+ 309us) server_negotiation.cc:299] Negotiation successful
0925 09:30:14.421700 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 09:30:14.422693 26666 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:30:14.459941 26719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:14.459428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:38917 (local address 127.0.0.1:60010)
0925 09:30:14.459656 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:14.459768 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260925 09:30:14.461050 26719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:14.460622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:38917 (local address 127.0.0.1:60022)
0925 09:30:14.460835 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:14.460909 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260925 09:30:14.517118 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 26543
I20260925 09:30:14.533834 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 26183
I20260925 09:30:14.535058 26245 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:30:14.641876 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 26183
I20260925 09:30:34.663363 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:40439
--webserver_interface=127.10.114.126
--webserver_port=33937
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:40439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:30:34.982491 26723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:30:34.982806 26723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:30:34.982952 26723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:30:34.989992 26723 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 09:30:34.990115 26723 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 09:30:34.990203 26723 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 09:30:34.990301 26723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:30:34.990388 26723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:30:34.990520 26723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:30:34.990592 26723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:30:34.990677 26723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:30:35.000058 26723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:40439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:40439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=33937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:30:35.001256 26723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:30:35.002629 26723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:30:35.025375 26730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:30:35.025533 26723 server_base.cc:1034] running on GCE node
W20260925 09:30:35.025319 26729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:35.025319 26732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:30:35.026765 26723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:30:35.028441 26723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:30:35.029691 26723 hybrid_clock.cc:648] HybridClock initialized: now 1790328635029617 us; error 50 us; skew 500 ppm
I20260925 09:30:35.030076 26723 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:30:35 dist-test-slave-9pj8 krb5kdc[26170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328635, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:30:35.033809 26723 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:30:35.035408 26723 webserver.cc:460] Webserver started at http://127.10.114.126:33937/ using document root <none> and password file <none>
I20260925 09:30:35.036228 26723 fs_manager.cc:359] Metadata directory not provided
I20260925 09:30:35.036367 26723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:30:35.042130 26723 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260925 09:30:35.044957 26739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:35.046150 26723 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:30:35.046332 26723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "a613cdc33a34404bbe16d80623c99f25"
format_stamp: "Formatted at 2026-09-25 09:30:11 on dist-test-slave-9pj8"
server_key: "3981ca57af12dd69e0328c8b4e2a56d9"
server_key_iv: "b4a130dbea1923bbda017dac5b3c516b"
server_key_version: "encryptionkey@0"
I20260925 09:30:35.047401 26723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:30:35.059181 26723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:30:35.063237 26723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:30:35.063498 26723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:30:35.082176 26723 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:40439
I20260925 09:30:35.082216 26790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:40439 every 8 connection(s)
I20260925 09:30:35.083571 26723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:30:35.084834 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 26723
I20260925 09:30:35.085624 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:38917
--local_ip_for_outbound_sockets=127.10.114.65
--tserver_master_addrs=127.10.114.126:40439
--webserver_port=39533
--webserver_interface=127.10.114.65
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:30:35.090571 26791 sys_catalog.cc:263] Verifying existing consensus state
I20260925 09:30:35.094094 26791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25: Bootstrap starting.
I20260925 09:30:35.123072 26791 log.cc:824] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25: Log is configured to *not* fsync() on all Append() calls
I20260925 09:30:35.138028 26791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 09:30:35.138636 26791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25: Bootstrap complete.
I20260925 09:30:35.152930 26791 raft_consensus.cc:348] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a613cdc33a34404bbe16d80623c99f25" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40439 } }
I20260925 09:30:35.154395 26791 raft_consensus.cc:729] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a613cdc33a34404bbe16d80623c99f25, State: Initialized, Role: FOLLOWER
I20260925 09:30:35.155121 26791 consensus_queue.cc:260] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a613cdc33a34404bbe16d80623c99f25" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40439 } }
I20260925 09:30:35.155426 26791 raft_consensus.cc:388] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:30:35.155556 26791 raft_consensus.cc:482] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:30:35.155743 26791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:30:35.161134 26791 raft_consensus.cc:504] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a613cdc33a34404bbe16d80623c99f25" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40439 } }
I20260925 09:30:35.161742 26791 leader_election.cc:302] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a613cdc33a34404bbe16d80623c99f25; no voters:
I20260925 09:30:35.162379 26791 leader_election.cc:288] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 09:30:35.162535 26795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:30:35.163918 26795 raft_consensus.cc:686] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 2 LEADER]: Becoming Leader. State: Replica: a613cdc33a34404bbe16d80623c99f25, State: Running, Role: LEADER
I20260925 09:30:35.164445 26795 consensus_queue.cc:237] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a613cdc33a34404bbe16d80623c99f25" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40439 } }
I20260925 09:30:35.164567 26791 sys_catalog.cc:564] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:30:35.166958 26797 sys_catalog.cc:455] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a613cdc33a34404bbe16d80623c99f25. Latest consensus state: current_term: 2 leader_uuid: "a613cdc33a34404bbe16d80623c99f25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a613cdc33a34404bbe16d80623c99f25" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40439 } } }
I20260925 09:30:35.167443 26797 sys_catalog.cc:458] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [sys.catalog]: This master's current role is: LEADER
I20260925 09:30:35.166978 26796 sys_catalog.cc:455] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a613cdc33a34404bbe16d80623c99f25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a613cdc33a34404bbe16d80623c99f25" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40439 } } }
I20260925 09:30:35.168013 26796 sys_catalog.cc:458] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [sys.catalog]: This master's current role is: LEADER
I20260925 09:30:35.169508 26802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:30:35.177238 26802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:30:35.178655 26802 catalog_manager.cc:1259] Loaded cluster ID: 12ef670e78cc4c03b0e96a79717fef37
I20260925 09:30:35.178757 26802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:30:35.182823 26802 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:30:35.184388 26802 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25: Loaded TSK: 0
I20260925 09:30:35.229809 26802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25: Generated new TSK 1
I20260925 09:30:35.230188 26802 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 09:30:35.466843 26793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:30:35.467133 26793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:30:35.467281 26793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:30:35.474097 26793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:30:35.474231 26793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:30:35.474381 26793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:30:35.483994 26793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:38917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=39533
--tserver_master_addrs=127.10.114.126:40439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:30:35.485134 26793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:30:35.486358 26793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:30:35.500326 26818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:35.500388 26819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:35.500991 26821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:30:35.501991 26793 server_base.cc:1034] running on GCE node
I20260925 09:30:35.502563 26793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:30:35.503448 26793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:30:35.504670 26793 hybrid_clock.cc:648] HybridClock initialized: now 1790328635504600 us; error 70 us; skew 500 ppm
I20260925 09:30:35.504962 26793 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:30:35 dist-test-slave-9pj8 krb5kdc[26170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328635, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:30:35.508077 26793 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:30:35.509573 26793 webserver.cc:460] Webserver started at http://127.10.114.65:39533/ using document root <none> and password file <none>
I20260925 09:30:35.510277 26793 fs_manager.cc:359] Metadata directory not provided
I20260925 09:30:35.510424 26793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:30:35.515383 26793 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:30:35.518067 26828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:35.519320 26793 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 09:30:35.519469 26793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "2a9f4e4a791044729f31d00e037e3845"
format_stamp: "Formatted at 2026-09-25 09:30:12 on dist-test-slave-9pj8"
server_key: "43fd67f0c87a53c63fa6a45f42b3f6cd"
server_key_iv: "e3d7fea6aac2ea4ea3ad896acd4c8c3c"
server_key_version: "encryptionkey@0"
I20260925 09:30:35.520447 26793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:30:35.536101 26793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:30:35.540942 26793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:30:35.541234 26793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:30:35.542418 26793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:30:35.544670 26835 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260925 09:30:35.550848 26835 ts_tablet_manager.cc:1852] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260925 09:30:35.552039 26835 ts_tablet_manager.cc:1905] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:30:35.561192 26835 ts_tablet_manager.cc:1918] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:30:35.561378 26835 log.cc:1196] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/28524542955a4c3dab832a6bc738f4cd
I20260925 09:30:35.561790 26835 ts_tablet_manager.cc:1939] T 28524542955a4c3dab832a6bc738f4cd P 2a9f4e4a791044729f31d00e037e3845: Deleting consensus metadata
I20260925 09:30:35.562247 26793 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260925 09:30:35.562368 26793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.018s user 0.001s sys 0.000s
I20260925 09:30:35.562517 26793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:30:35.562609 26793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:35.600302 26793 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:38917
I20260925 09:30:35.600361 26941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:38917 every 8 connection(s)
I20260925 09:30:35.601763 26793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:30:35.611730 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 26793
I20260925 09:30:35.612567 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:33579
--local_ip_for_outbound_sockets=127.10.114.66
--tserver_master_addrs=127.10.114.126:40439
--webserver_port=37829
--webserver_interface=127.10.114.66
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:30:35 dist-test-slave-9pj8 krb5kdc[26170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328635, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:30:35.628225 26944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:35.604847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:51301 (local address 127.10.114.126:40439)
0925 09:30:35.605497 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:35.605509 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:35.606146 (+ 637us) server_negotiation.cc:407] Connection header received
0925 09:30:35.608359 (+ 2213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:35.608373 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:35.608737 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:35.609053 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:30:35.610448 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:35.611959 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:35.614484 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:35.615241 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:35.615397 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:35.619825 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:30:35.619840 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:30:35.619854 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:30:35.619891 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:30:35.623753 (+ 3862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:35.624611 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:35.624620 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:35.624626 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:35.624794 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:35.625289 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:35.625296 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:35.625300 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:35.625758 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:30:35.625943 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:35.626642 (+ 699us) server_negotiation.cc:299] Negotiation successful
0925 09:30:35.627004 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":265,"threads_started":1}
I20260925 09:30:35.628216 26945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:35.605008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.65:51301)
0925 09:30:35.605936 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:35.605987 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 09:30:35.607815 (+ 1828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:30:35.609216 (+ 1401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:30:35.609223 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:30:35.609732 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:30:35.610259 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:35.610275 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:35.612138 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:35.612144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:35.614311 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:35.614333 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:35.615476 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:35.615485 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:35.615808 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:35.616560 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:30:35.616591 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:30:35.619530 (+ 2939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:30:35.623947 (+ 4417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:30:35.623953 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:30:35.623964 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:30:35.624473 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:30:35.625005 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:30:35.625010 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:30:35.625014 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:30:35.625181 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:30:35.625880 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:30:35.625887 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:30:35.626200 (+ 313us) client_negotiation.cc:769] Sending connection context
0925 09:30:35.626439 (+ 239us) client_negotiation.cc:240] Negotiation successful
0925 09:30:35.626941 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":409,"threads_started":1}
I20260925 09:30:35.631517 26942 heartbeater.cc:347] Connected to a master server at 127.10.114.126:40439
I20260925 09:30:35.631842 26942 heartbeater.cc:464] Registering TS with master...
I20260925 09:30:35.632467 26942 heartbeater.cc:511] Master 127.10.114.126:40439 requested a full tablet report, sending...
I20260925 09:30:35.635007 26756 ts_manager.cc:194] Registered new tserver with Master: 2a9f4e4a791044729f31d00e037e3845 (127.10.114.65:38917)
I20260925 09:30:35.637517 26756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:51301
W20260925 09:30:35.939318 26946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:30:35.939569 26946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:30:35.939668 26946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:30:35.946566 26946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:30:35.946666 26946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:30:35.946846 26946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:30:35.956421 26946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:33579
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=37829
--tserver_master_addrs=127.10.114.126:40439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:30:35.957533 26946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:30:35.958742 26946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:30:35.971491 26954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:35.971449 26952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:35.971515 26951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:30:35.972152 26946 server_base.cc:1034] running on GCE node
I20260925 09:30:35.972723 26946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:30:35.973558 26946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:30:35.974742 26946 hybrid_clock.cc:648] HybridClock initialized: now 1790328635974702 us; error 35 us; skew 500 ppm
I20260925 09:30:35.975029 26946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:30:35 dist-test-slave-9pj8 krb5kdc[26170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328635, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:30:35.978091 26946 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:30:35.979477 26946 webserver.cc:460] Webserver started at http://127.10.114.66:37829/ using document root <none> and password file <none>
I20260925 09:30:35.980140 26946 fs_manager.cc:359] Metadata directory not provided
I20260925 09:30:35.980249 26946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:30:35.985096 26946 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260925 09:30:35.987536 26961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:35.988895 26946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:30:35.989028 26946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "ab1d10dc32c3439f8dab35ca8c44afe2"
format_stamp: "Formatted at 2026-09-25 09:30:12 on dist-test-slave-9pj8"
server_key: "0e992f59ce9eb8f8da441b4bda81ecd3"
server_key_iv: "40557a36f9ea460098fdd6125cefef0e"
server_key_version: "encryptionkey@0"
I20260925 09:30:35.989957 26946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:30:36.003185 26946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:30:36.006784 26946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:30:36.006999 26946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:30:36.007848 26946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:30:36.009050 26946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:30:36.009136 26946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:36.009214 26946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:30:36.009269 26946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:36.041782 26946 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:33579
I20260925 09:30:36.041858 27073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:33579 every 8 connection(s)
I20260925 09:30:36.043187 26946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:30:36.049009 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 26946
I20260925 09:30:36.049651 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:40219
--local_ip_for_outbound_sockets=127.10.114.67
--tserver_master_addrs=127.10.114.126:40439
--webserver_port=39227
--webserver_interface=127.10.114.67
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:30:36 dist-test-slave-9pj8 krb5kdc[26170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328635, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:30:36.062330 26944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.045561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:56983 (local address 127.10.114.126:40439)
0925 09:30:36.045734 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:36.045738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:36.046880 (+ 1142us) server_negotiation.cc:407] Connection header received
0925 09:30:36.048696 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:36.048701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:36.048784 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:36.048887 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:30:36.050592 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.051365 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.052929 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.053341 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.053475 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.056772 (+ 3297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:30:36.056781 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:30:36.056786 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:30:36.056831 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:30:36.059532 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:36.060267 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:36.060272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:36.060275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:36.060390 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:36.060790 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:36.060794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:36.060797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:36.061043 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:30:36.061185 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:36.061790 (+ 605us) server_negotiation.cc:299] Negotiation successful
0925 09:30:36.062010 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 09:30:36.063274 27076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.046044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.66:56983)
0925 09:30:36.046730 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:36.046771 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 09:30:36.048438 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:30:36.049245 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:30:36.049257 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:30:36.049794 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:30:36.050436 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:36.050454 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.051503 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.051508 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:36.052795 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:36.052806 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.053552 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.053560 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:36.053765 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.054362 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:30:36.054382 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:30:36.056549 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:30:36.059664 (+ 3115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:30:36.059674 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:30:36.059688 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:30:36.060144 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:30:36.060494 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:30:36.060500 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:30:36.060503 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:30:36.060671 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:30:36.061176 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:30:36.061183 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:30:36.061474 (+ 291us) client_negotiation.cc:769] Sending connection context
0925 09:30:36.061702 (+ 228us) client_negotiation.cc:240] Negotiation successful
0925 09:30:36.062171 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":273,"threads_started":1}
I20260925 09:30:36.064678 27074 heartbeater.cc:347] Connected to a master server at 127.10.114.126:40439
I20260925 09:30:36.064957 27074 heartbeater.cc:464] Registering TS with master...
I20260925 09:30:36.065660 27074 heartbeater.cc:511] Master 127.10.114.126:40439 requested a full tablet report, sending...
I20260925 09:30:36.067150 26756 ts_manager.cc:194] Registered new tserver with Master: ab1d10dc32c3439f8dab35ca8c44afe2 (127.10.114.66:33579)
I20260925 09:30:36.068161 26756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:56983
W20260925 09:30:36.370692 27077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:30:36.370994 27077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:30:36.371184 27077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:30:36.377929 27077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:30:36.378039 27077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:30:36.378244 27077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:30:36.387428 27077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:40219
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=39227
--tserver_master_addrs=127.10.114.126:40439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:30:36.388599 27077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:30:36.389879 27077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:30:36.402662 27082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:36.402638 27085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:36.402597 27083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:30:36.403401 27077 server_base.cc:1034] running on GCE node
I20260925 09:30:36.404011 27077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:30:36.404897 27077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:30:36.406095 27077 hybrid_clock.cc:648] HybridClock initialized: now 1790328636406058 us; error 29 us; skew 500 ppm
I20260925 09:30:36.406494 27077 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:30:36 dist-test-slave-9pj8 krb5kdc[26170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328636, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:30:36.409430 27077 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:30:36.410701 27077 webserver.cc:460] Webserver started at http://127.10.114.67:39227/ using document root <none> and password file <none>
I20260925 09:30:36.411337 27077 fs_manager.cc:359] Metadata directory not provided
I20260925 09:30:36.411484 27077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:30:36.416412 27077 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260925 09:30:36.418932 27092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:36.420205 27077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:30:36.420365 27077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "d8fea1757f77441b9115d8680c4f085d"
format_stamp: "Formatted at 2026-09-25 09:30:13 on dist-test-slave-9pj8"
server_key: "d3e50694b23a184bbf5096de9ddb0d94"
server_key_iv: "a6fd0ef9722b278a3be7b529789dac4f"
server_key_version: "encryptionkey@0"
I20260925 09:30:36.421310 27077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:30:36.448603 27077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:30:36.452827 27077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:30:36.453100 27077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:30:36.453960 27077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:30:36.455133 27077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:30:36.455260 27077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:36.455372 27077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:30:36.455458 27077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:36.485155 27077 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:40219
I20260925 09:30:36.485221 27204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:40219 every 8 connection(s)
I20260925 09:30:36.486568 27077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:30:36.492765 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 27077
Sep 25 09:30:36 dist-test-slave-9pj8 krb5kdc[26170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328636, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:30:36.509635 26944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.489103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:50551 (local address 127.10.114.126:40439)
0925 09:30:36.489301 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:36.489305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:36.490391 (+ 1086us) server_negotiation.cc:407] Connection header received
0925 09:30:36.492206 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:36.492211 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:36.492300 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:36.492423 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:30:36.494254 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.495015 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.496621 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.497057 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.497182 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.501625 (+ 4443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:30:36.501632 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:30:36.501666 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:30:36.501749 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:30:36.505991 (+ 4242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:36.506903 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:36.506910 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:36.506913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:36.507061 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:36.507600 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:36.507605 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:36.507608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:36.507986 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:30:36.508142 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:36.508957 (+ 815us) server_negotiation.cc:299] Negotiation successful
0925 09:30:36.509166 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 09:30:36.509720 27208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.493677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49914 (local address 127.10.114.126:40439)
0925 09:30:36.494200 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:36.494205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:36.494444 (+ 239us) server_negotiation.cc:407] Connection header received
0925 09:30:36.494624 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:36.494633 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:36.494697 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:36.494801 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:30:36.496351 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.497097 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.498952 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.499652 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.499795 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.502871 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:30:36.502877 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:30:36.502881 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:30:36.502937 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:30:36.506652 (+ 3715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:36.507524 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:36.507529 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:36.507533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:36.507669 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:36.508190 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:36.508194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:36.508197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:36.508513 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:30:36.508762 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:36.509096 (+ 334us) server_negotiation.cc:299] Negotiation successful
0925 09:30:36.509305 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":230,"threads_started":1}
I20260925 09:30:36.510629 27207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.489604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.67:50551)
0925 09:30:36.490250 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:36.490284 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 09:30:36.491967 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:30:36.492650 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:30:36.492658 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:30:36.493252 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:30:36.494043 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:36.494077 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.495247 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.495252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:36.496477 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:36.496488 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.497259 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.497268 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:36.497492 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.498215 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:30:36.498265 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:30:36.501365 (+ 3100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:30:36.506162 (+ 4797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:30:36.506173 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:30:36.506188 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:30:36.506786 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:30:36.507220 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:30:36.507227 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:30:36.507233 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:30:36.507458 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:30:36.508131 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:30:36.508139 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:30:36.508538 (+ 399us) client_negotiation.cc:769] Sending connection context
0925 09:30:36.508830 (+ 292us) client_negotiation.cc:240] Negotiation successful
0925 09:30:36.509312 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":256,"threads_started":1}
I20260925 09:30:36.511883 27205 heartbeater.cc:347] Connected to a master server at 127.10.114.126:40439
I20260925 09:30:36.512180 27205 heartbeater.cc:464] Registering TS with master...
I20260925 09:30:36.512766 27205 heartbeater.cc:511] Master 127.10.114.126:40439 requested a full tablet report, sending...
I20260925 09:30:36.513979 26756 ts_manager.cc:194] Registered new tserver with Master: d8fea1757f77441b9115d8680c4f085d (127.10.114.67:40219)
I20260925 09:30:36.514719 26756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:50551
I20260925 09:30:36.524420 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:30:36.530797 27208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.525276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49928 (local address 127.10.114.126:40439)
0925 09:30:36.525460 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:36.525468 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:36.525983 (+ 515us) server_negotiation.cc:407] Connection header received
0925 09:30:36.526212 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:36.526219 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:36.526364 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:36.526471 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:30:36.526846 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.527566 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.529001 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.529417 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.529517 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.529980 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:30:36.530195 (+ 215us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0925 09:30:36.530496 (+ 301us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49928: token expired
Metrics: {"server-negotiator.queue_time_us":51}
W20260925 09:30:36.530936 27208 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49928: token expired
W20260925 09:30:36.531436 26677 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.114.126:40439)
I20260925 09:30:36.543087 27208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.531890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49934 (local address 127.10.114.126:40439)
0925 09:30:36.532056 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:36.532060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:36.532077 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:30:36.532193 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:36.532197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:36.532249 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:36.532330 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:30:36.533552 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.534260 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.535434 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.535903 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.536003 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.538537 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:30:36.538542 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:30:36.538545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:30:36.538581 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:30:36.540973 (+ 2392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:36.541515 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:36.541518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:36.541521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:36.541632 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:36.541922 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:36.541925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:36.541927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:36.542201 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:30:36.542377 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:36.542620 (+ 243us) server_negotiation.cc:299] Negotiation successful
0925 09:30:36.542775 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 09:30:36.546056 26756 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49934:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:30:36.553380 26756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:30:36.576982 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.566796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:40219 (local address 127.0.0.1:44040)
0925 09:30:36.567416 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:36.567445 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 09:30:36.567573 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:30:36.569464 (+ 1891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:30:36.569472 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:30:36.569511 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:30:36.569953 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:36.569970 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.571952 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.571958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:36.573978 (+ 2020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:36.573996 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.575839 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.575848 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:36.576058 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.576145 (+ 87us) client_negotiation.cc:769] Sending connection context
0925 09:30:36.576255 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 09:30:36.576429 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":2,"thread_start_us":2087,"threads_started":1}
I20260925 09:30:36.577553 27215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.565728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:38917 (local address 127.0.0.1:58756)
0925 09:30:36.567032 (+ 1304us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:36.567083 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 09:30:36.567358 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:30:36.568941 (+ 1583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:30:36.568949 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:30:36.568982 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:30:36.569509 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:36.569523 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.571569 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.571574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:36.574327 (+ 2753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:36.574340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.576537 (+ 2197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.576543 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:36.576825 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.576869 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 09:30:36.576981 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 09:30:36.577126 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"mutex_wait_us":109,"thread_start_us":343,"threads_started":1}
I20260925 09:30:36.577802 27218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.567432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:33579 (local address 127.0.0.1:57730)
0925 09:30:36.568076 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:36.568093 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:30:36.568207 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:30:36.570827 (+ 2620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:30:36.570832 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:30:36.570857 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:30:36.571180 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:36.571191 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.573001 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.573012 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:36.574853 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:36.574863 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.576596 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.576602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:36.576824 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.576850 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 09:30:36.576952 (+ 102us) client_negotiation.cc:240] Negotiation successful
0925 09:30:36.577068 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"mutex_wait_us":20,"thread_start_us":1290,"threads_started":1}
I20260925 09:30:36.577829 27220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.568661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57730 (local address 127.10.114.66:33579)
0925 09:30:36.570281 (+ 1620us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:36.570290 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:36.570318 (+ 28us) server_negotiation.cc:407] Connection header received
0925 09:30:36.570415 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:36.570423 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:36.570629 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:36.570822 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:30:36.571341 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.572868 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.574989 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.576368 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.576514 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.576692 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:36.576992 (+ 300us) server_negotiation.cc:299] Negotiation successful
0925 09:30:36.577429 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1472,"thread_start_us":1322,"threads_started":1}
I20260925 09:30:36.578276 27216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.565851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58756 (local address 127.10.114.65:38917)
0925 09:30:36.568255 (+ 2404us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:36.568267 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:36.568300 (+ 33us) server_negotiation.cc:407] Connection header received
0925 09:30:36.568408 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:36.568416 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:36.568675 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:36.569782 (+ 1107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:30:36.570030 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.571447 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.574486 (+ 3039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.576030 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.576158 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.576361 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:36.577795 (+ 1434us) server_negotiation.cc:299] Negotiation successful
0925 09:30:36.577947 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2231,"thread_start_us":2108,"threads_started":1}
I20260925 09:30:36.578305 27219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.568006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44040 (local address 127.10.114.67:40219)
0925 09:30:36.568756 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:36.568765 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:36.568799 (+ 34us) server_negotiation.cc:407] Connection header received
0925 09:30:36.568914 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:36.568924 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:36.569191 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:36.570148 (+ 957us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:30:36.570395 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.571811 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.574144 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.575531 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.577348 (+ 1817us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.577583 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:36.577772 (+ 189us) server_negotiation.cc:299] Negotiation successful
0925 09:30:36.577933 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":1670,"threads_started":1}
I20260925 09:30:36.581359 27009 tablet_service.cc:1467] Processing CreateTablet for tablet b0c772044de949169d165a4fe954beb1 (DEFAULT_TABLE table=test-table [id=04c4568b47624f35aa784cfea6395031]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:30:36.581341 26877 tablet_service.cc:1467] Processing CreateTablet for tablet b0c772044de949169d165a4fe954beb1 (DEFAULT_TABLE table=test-table [id=04c4568b47624f35aa784cfea6395031]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:30:36.581761 27140 tablet_service.cc:1467] Processing CreateTablet for tablet b0c772044de949169d165a4fe954beb1 (DEFAULT_TABLE table=test-table [id=04c4568b47624f35aa784cfea6395031]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:30:36.582610 26877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0c772044de949169d165a4fe954beb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:30:36.582602 27009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0c772044de949169d165a4fe954beb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:30:36.582958 27140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0c772044de949169d165a4fe954beb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:30:36.593647 27221 tablet_bootstrap.cc:492] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2: Bootstrap starting.
I20260925 09:30:36.596773 27221 tablet_bootstrap.cc:654] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2: Neither blocks nor log segments found. Creating new log.
I20260925 09:30:36.598033 27221 log.cc:824] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2: Log is configured to *not* fsync() on all Append() calls
I20260925 09:30:36.600113 27224 tablet_bootstrap.cc:492] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d: Bootstrap starting.
I20260925 09:30:36.604854 27224 tablet_bootstrap.cc:654] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d: Neither blocks nor log segments found. Creating new log.
I20260925 09:30:36.606720 27222 tablet_bootstrap.cc:492] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845: Bootstrap starting.
I20260925 09:30:36.606441 27224 log.cc:824] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d: Log is configured to *not* fsync() on all Append() calls
I20260925 09:30:36.608754 27221 tablet_bootstrap.cc:492] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2: No bootstrap required, opened a new log
I20260925 09:30:36.609056 27221 ts_tablet_manager.cc:1397] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.006s
I20260925 09:30:36.611426 27222 tablet_bootstrap.cc:654] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845: Neither blocks nor log segments found. Creating new log.
I20260925 09:30:36.612776 27222 log.cc:824] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845: Log is configured to *not* fsync() on all Append() calls
I20260925 09:30:36.614329 27224 tablet_bootstrap.cc:492] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d: No bootstrap required, opened a new log
I20260925 09:30:36.614593 27224 ts_tablet_manager.cc:1397] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.004s
I20260925 09:30:36.615729 27222 tablet_bootstrap.cc:492] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845: No bootstrap required, opened a new log
I20260925 09:30:36.615998 27222 ts_tablet_manager.cc:1397] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260925 09:30:36.617605 27221 raft_consensus.cc:348] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } } peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } }
I20260925 09:30:36.618130 27221 raft_consensus.cc:374] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:30:36.618278 27221 raft_consensus.cc:729] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab1d10dc32c3439f8dab35ca8c44afe2, State: Initialized, Role: FOLLOWER
I20260925 09:30:36.618854 27221 consensus_queue.cc:260] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } } peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } }
I20260925 09:30:36.619939 27074 heartbeater.cc:503] Master 127.10.114.126:40439 was elected leader, sending a full tablet report...
I20260925 09:30:36.620560 27221 ts_tablet_manager.cc:1428] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2: Time spent starting tablet: real 0.011s user 0.008s sys 0.002s
I20260925 09:30:36.625473 27224 raft_consensus.cc:348] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } } peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } }
I20260925 09:30:36.625906 27224 raft_consensus.cc:374] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:30:36.626008 27224 raft_consensus.cc:729] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8fea1757f77441b9115d8680c4f085d, State: Initialized, Role: FOLLOWER
I20260925 09:30:36.626564 27224 consensus_queue.cc:260] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } } peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } }
I20260925 09:30:36.626792 27222 raft_consensus.cc:348] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } } peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } }
I20260925 09:30:36.627256 27222 raft_consensus.cc:374] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:30:36.627367 27222 raft_consensus.cc:729] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9f4e4a791044729f31d00e037e3845, State: Initialized, Role: FOLLOWER
I20260925 09:30:36.627564 27205 heartbeater.cc:503] Master 127.10.114.126:40439 was elected leader, sending a full tablet report...
I20260925 09:30:36.628001 27222 consensus_queue.cc:260] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } } peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } }
I20260925 09:30:36.628334 27224 ts_tablet_manager.cc:1428] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
I20260925 09:30:36.629402 26942 heartbeater.cc:503] Master 127.10.114.126:40439 was elected leader, sending a full tablet report...
I20260925 09:30:36.630131 27222 ts_tablet_manager.cc:1428] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260925 09:30:36.738265 27206 tablet.cc:2369] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:30:36.795080 27075 tablet.cc:2369] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:30:36.854398 26943 tablet.cc:2369] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:30:36.900465 27227 raft_consensus.cc:482] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:30:36.900496 27229 raft_consensus.cc:482] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:30:36.900771 27227 raft_consensus.cc:504] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } } peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } }
I20260925 09:30:36.900785 27229 raft_consensus.cc:504] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } } peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } }
I20260925 09:30:36.902344 27227 leader_election.cc:288] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9f4e4a791044729f31d00e037e3845 (127.10.114.65:38917), d8fea1757f77441b9115d8680c4f085d (127.10.114.67:40219)
I20260925 09:30:36.903818 27229 leader_election.cc:288] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8fea1757f77441b9115d8680c4f085d (127.10.114.67:40219), ab1d10dc32c3439f8dab35ca8c44afe2 (127.10.114.66:33579)
I20260925 09:30:36.912127 27219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.902903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:37627 (local address 127.10.114.67:40219)
0925 09:30:36.903169 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:36.903177 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:36.903968 (+ 791us) server_negotiation.cc:407] Connection header received
0925 09:30:36.904422 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:36.904430 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:36.904520 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:36.904586 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:30:36.905450 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.906628 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.909530 (+ 2902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.910866 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.911014 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.911094 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:36.911674 (+ 580us) server_negotiation.cc:299] Negotiation successful
0925 09:30:36.911852 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":78}
I20260925 09:30:36.912889 27220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.903229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:46843 (local address 127.10.114.66:33579)
0925 09:30:36.903464 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:36.903472 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:36.904502 (+ 1030us) server_negotiation.cc:407] Connection header received
0925 09:30:36.904738 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:36.904747 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:36.904823 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:36.904888 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:30:36.905680 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.906788 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.910338 (+ 3550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.911610 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.911769 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.911855 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:36.912491 (+ 636us) server_negotiation.cc:299] Negotiation successful
0925 09:30:36.912624 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 09:30:36.912983 27216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.904464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:33039 (local address 127.10.114.65:38917)
0925 09:30:36.904690 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:36.904698 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:36.905328 (+ 630us) server_negotiation.cc:407] Connection header received
0925 09:30:36.905586 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:36.905591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:36.905666 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:36.905726 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:30:36.906689 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.908304 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.910741 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.911921 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.912051 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.912140 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:36.912609 (+ 469us) server_negotiation.cc:299] Negotiation successful
0925 09:30:36.912723 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":63}
I20260925 09:30:36.913288 27230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.902619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:40219 (local address 127.10.114.65:37627)
0925 09:30:36.903836 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:36.903858 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:30:36.904212 (+ 354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:30:36.904810 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:30:36.904815 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:30:36.904838 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:30:36.905263 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:36.905281 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.906850 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.906856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:36.909356 (+ 2500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:36.909373 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.911282 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.911290 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:36.911483 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.911520 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 09:30:36.912505 (+ 985us) client_negotiation.cc:240] Negotiation successful
0925 09:30:36.912634 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1048,"mutex_wait_us":37,"thread_start_us":848,"threads_started":1}
I20260925 09:30:36.913583 27231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.902693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:38917 (local address 127.10.114.66:33039)
0925 09:30:36.905239 (+ 2546us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:36.905257 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:30:36.905423 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:30:36.905942 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:30:36.905950 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:30:36.905986 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:30:36.906485 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:36.906499 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.908411 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.908419 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:36.910569 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:36.910582 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.912273 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.912279 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:36.912456 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.912487 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 09:30:36.913123 (+ 636us) client_negotiation.cc:240] Negotiation successful
0925 09:30:36.913255 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2417,"thread_start_us":2216,"threads_started":1}
I20260925 09:30:36.913690 27232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.902732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:33579 (local address 127.10.114.65:46843)
0925 09:30:36.904413 (+ 1681us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:36.904438 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 09:30:36.904586 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:30:36.905102 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:30:36.905108 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:30:36.905131 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:30:36.905499 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:36.905511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.906956 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.906963 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:36.910163 (+ 3200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:36.910179 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.912065 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.912071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:36.912325 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.912355 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 09:30:36.913136 (+ 781us) client_negotiation.cc:240] Negotiation successful
0925 09:30:36.913255 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1558,"mutex_wait_us":993,"thread_start_us":1397,"threads_started":1}
I20260925 09:30:36.914512 27029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0c772044de949169d165a4fe954beb1" candidate_uuid: "2a9f4e4a791044729f31d00e037e3845" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" is_pre_election: true
I20260925 09:30:36.914533 27160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0c772044de949169d165a4fe954beb1" candidate_uuid: "2a9f4e4a791044729f31d00e037e3845" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8fea1757f77441b9115d8680c4f085d" is_pre_election: true
I20260925 09:30:36.914676 26897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0c772044de949169d165a4fe954beb1" candidate_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9f4e4a791044729f31d00e037e3845" is_pre_election: true
I20260925 09:30:36.914997 27029 raft_consensus.cc:2454] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9f4e4a791044729f31d00e037e3845 in term 0.
I20260925 09:30:36.915076 27160 raft_consensus.cc:2454] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a9f4e4a791044729f31d00e037e3845 in term 0.
I20260925 09:30:36.915104 26897 raft_consensus.cc:2454] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab1d10dc32c3439f8dab35ca8c44afe2 in term 0.
I20260925 09:30:36.915668 26829 leader_election.cc:302] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9f4e4a791044729f31d00e037e3845, d8fea1757f77441b9115d8680c4f085d; no voters:
I20260925 09:30:36.915861 26962 leader_election.cc:302] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9f4e4a791044729f31d00e037e3845, ab1d10dc32c3439f8dab35ca8c44afe2; no voters:
I20260925 09:30:36.916281 27233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.905349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:40219 (local address 127.10.114.66:54435)
0925 09:30:36.907881 (+ 2532us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:36.907898 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:30:36.908002 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:30:36.909008 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:30:36.909013 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:30:36.909039 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:30:36.909485 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:36.909504 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.911090 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.911095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:36.913159 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:30:36.913175 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.915284 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.915290 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:30:36.915474 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.915503 (+ 29us) client_negotiation.cc:769] Sending connection context
0925 09:30:36.915585 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 09:30:36.915687 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2391,"thread_start_us":2217,"threads_started":1}
I20260925 09:30:36.916355 27234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:36.905666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:54435 (local address 127.10.114.67:40219)
0925 09:30:36.908543 (+ 2877us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:36.908548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:36.908569 (+ 21us) server_negotiation.cc:407] Connection header received
0925 09:30:36.908616 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:36.908621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:36.908679 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:36.908768 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:30:36.909693 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.910945 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.913344 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:36.914878 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:36.915047 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:36.915114 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:36.915930 (+ 816us) server_negotiation.cc:299] Negotiation successful
0925 09:30:36.916049 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2746,"thread_start_us":2551,"threads_started":1}
I20260925 09:30:36.916337 27229 raft_consensus.cc:2781] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:30:36.916455 27229 raft_consensus.cc:482] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:30:36.916236 27227 raft_consensus.cc:2781] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:30:36.916556 27229 raft_consensus.cc:3037] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:30:36.916630 27227 raft_consensus.cc:482] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:30:36.916747 27227 raft_consensus.cc:3037] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:30:36.917124 27160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0c772044de949169d165a4fe954beb1" candidate_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8fea1757f77441b9115d8680c4f085d" is_pre_election: true
I20260925 09:30:36.917323 27160 raft_consensus.cc:2454] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab1d10dc32c3439f8dab35ca8c44afe2 in term 0.
I20260925 09:30:36.919765 27229 raft_consensus.cc:504] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } } peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } }
I20260925 09:30:36.919765 27227 raft_consensus.cc:504] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } } peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } }
I20260925 09:30:36.920425 27227 leader_election.cc:288] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2 [CANDIDATE]: Term 1 election: Requested vote from peers 2a9f4e4a791044729f31d00e037e3845 (127.10.114.65:38917), d8fea1757f77441b9115d8680c4f085d (127.10.114.67:40219)
I20260925 09:30:36.920931 27160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0c772044de949169d165a4fe954beb1" candidate_uuid: "2a9f4e4a791044729f31d00e037e3845" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8fea1757f77441b9115d8680c4f085d"
I20260925 09:30:36.920941 26897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0c772044de949169d165a4fe954beb1" candidate_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9f4e4a791044729f31d00e037e3845"
I20260925 09:30:36.920941 27029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0c772044de949169d165a4fe954beb1" candidate_uuid: "2a9f4e4a791044729f31d00e037e3845" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2"
I20260925 09:30:36.921135 27160 raft_consensus.cc:3037] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:30:36.921201 27029 raft_consensus.cc:2379] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2a9f4e4a791044729f31d00e037e3845 in current term 1: Already voted for candidate ab1d10dc32c3439f8dab35ca8c44afe2 in this term.
I20260925 09:30:36.921217 26897 raft_consensus.cc:2379] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ab1d10dc32c3439f8dab35ca8c44afe2 in current term 1: Already voted for candidate 2a9f4e4a791044729f31d00e037e3845 in this term.
I20260925 09:30:36.921620 27229 leader_election.cc:288] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [CANDIDATE]: Term 1 election: Requested vote from peers d8fea1757f77441b9115d8680c4f085d (127.10.114.67:40219), ab1d10dc32c3439f8dab35ca8c44afe2 (127.10.114.66:33579)
I20260925 09:30:36.921783 27159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0c772044de949169d165a4fe954beb1" candidate_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8fea1757f77441b9115d8680c4f085d"
I20260925 09:30:36.922477 26962 leader_election.cc:302] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ab1d10dc32c3439f8dab35ca8c44afe2; no voters: 2a9f4e4a791044729f31d00e037e3845, d8fea1757f77441b9115d8680c4f085d
I20260925 09:30:36.922720 27227 raft_consensus.cc:2726] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 09:30:36.923897 27160 raft_consensus.cc:2454] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a9f4e4a791044729f31d00e037e3845 in term 1.
I20260925 09:30:36.924229 26829 leader_election.cc:302] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a9f4e4a791044729f31d00e037e3845, d8fea1757f77441b9115d8680c4f085d; no voters: ab1d10dc32c3439f8dab35ca8c44afe2
I20260925 09:30:36.924522 27229 raft_consensus.cc:2781] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:30:36.924952 27229 raft_consensus.cc:686] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [term 1 LEADER]: Becoming Leader. State: Replica: 2a9f4e4a791044729f31d00e037e3845, State: Running, Role: LEADER
I20260925 09:30:36.925449 27229 consensus_queue.cc:237] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } } peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } }
I20260925 09:30:36.929591 26755 catalog_manager.cc:5495] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a9f4e4a791044729f31d00e037e3845 (127.10.114.65). New cstate: current_term: 1 leader_uuid: "2a9f4e4a791044729f31d00e037e3845" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a9f4e4a791044729f31d00e037e3845" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 38917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 } health_report { overall_health: UNKNOWN } } }
I20260925 09:30:37.032280 27216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:37.026772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58762 (local address 127.10.114.65:38917)
0925 09:30:37.026937 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:37.026944 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:37.027024 (+ 80us) server_negotiation.cc:407] Connection header received
0925 09:30:37.027260 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:37.027263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:37.027350 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:37.027494 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:30:37.027890 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:37.028671 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:37.030203 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:37.030619 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:37.030751 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:37.031223 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:30:37.031687 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:30:37.031864 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:37.031937 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 09:30:37.032011 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 09:30:37.039892 27160 raft_consensus.cc:1261] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d [term 1 FOLLOWER]: Refusing update from remote peer 2a9f4e4a791044729f31d00e037e3845: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:30:37.039888 27029 raft_consensus.cc:1261] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2 [term 1 FOLLOWER]: Refusing update from remote peer 2a9f4e4a791044729f31d00e037e3845: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:30:37.040756 27235 consensus_queue.cc:1035] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8fea1757f77441b9115d8680c4f085d" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:30:37.041111 27229 consensus_queue.cc:1035] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab1d10dc32c3439f8dab35ca8c44afe2" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 33579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:30:37.056288 27238 mvcc.cc:205] Tried to move back new op lower bound from 7333186097306017792 to 7333186096850239488. Current Snapshot: MvccSnapshot[applied={T|T < 7333186097306017792}]
I20260925 09:30:37.078411 26756 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:30:37.078743 26756 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:30:37.082592 26756 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25: Sending DeleteTablet for 3 replicas of tablet b0c772044de949169d165a4fe954beb1
I20260925 09:30:37.083771 27140 tablet_service.cc:1514] Processing DeleteTablet for tablet b0c772044de949169d165a4fe954beb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:30:37 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:44040
I20260925 09:30:37.084038 27009 tablet_service.cc:1514] Processing DeleteTablet for tablet b0c772044de949169d165a4fe954beb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:30:37 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:57730
I20260925 09:30:37.084803 26877 tablet_service.cc:1514] Processing DeleteTablet for tablet b0c772044de949169d165a4fe954beb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:30:37 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:58756
I20260925 09:30:37.087121 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 26793
I20260925 09:30:37.092258 27249 tablet_replica.cc:330] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2: stopping tablet replica
I20260925 09:30:37.092640 27249 raft_consensus.cc:2229] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:30:37.094270 27249 raft_consensus.cc:2258] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:30:37.097604 27249 ts_tablet_manager.cc:1905] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:30:37.104992 27249 ts_tablet_manager.cc:1918] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:30:37.105271 27249 log.cc:1196] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/b0c772044de949169d165a4fe954beb1
I20260925 09:30:37.105937 27249 ts_tablet_manager.cc:1939] T b0c772044de949169d165a4fe954beb1 P ab1d10dc32c3439f8dab35ca8c44afe2: Deleting consensus metadata
I20260925 09:30:37.107973 26742 catalog_manager.cc:4841] TS ab1d10dc32c3439f8dab35ca8c44afe2 (127.10.114.66:33579): tablet b0c772044de949169d165a4fe954beb1 (table test-table [id=04c4568b47624f35aa784cfea6395031]) successfully deleted
I20260925 09:30:37.113391 27248 tablet_replica.cc:330] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d: stopping tablet replica
I20260925 09:30:37.114604 27248 raft_consensus.cc:2229] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:30:37.117362 27250 tablet_replica.cc:330] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845: stopping tablet replica
I20260925 09:30:37.117789 27250 raft_consensus.cc:2229] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:30:37.118480 27250 raft_consensus.cc:2258] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:30:37.119156 27248 raft_consensus.cc:2258] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:30:37.121063 27250 ts_tablet_manager.cc:1905] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:30:37.121083 27248 ts_tablet_manager.cc:1905] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:30:37.127259 27248 ts_tablet_manager.cc:1918] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:30:37.127460 27248 log.cc:1196] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/b0c772044de949169d165a4fe954beb1
I20260925 09:30:37.128103 27248 ts_tablet_manager.cc:1939] T b0c772044de949169d165a4fe954beb1 P d8fea1757f77441b9115d8680c4f085d: Deleting consensus metadata
I20260925 09:30:37.128571 27250 ts_tablet_manager.cc:1918] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:30:37.128726 27250 log.cc:1196] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/b0c772044de949169d165a4fe954beb1
I20260925 09:30:37.129573 27250 ts_tablet_manager.cc:1939] T b0c772044de949169d165a4fe954beb1 P 2a9f4e4a791044729f31d00e037e3845: Deleting consensus metadata
I20260925 09:30:37.130468 26740 catalog_manager.cc:4841] TS d8fea1757f77441b9115d8680c4f085d (127.10.114.67:40219): tablet b0c772044de949169d165a4fe954beb1 (table test-table [id=04c4568b47624f35aa784cfea6395031]) successfully deleted
I20260925 09:30:37.131510 26740 catalog_manager.cc:4841] TS 2a9f4e4a791044729f31d00e037e3845 (127.10.114.65:38917): tablet b0c772044de949169d165a4fe954beb1 (table test-table [id=04c4568b47624f35aa784cfea6395031]) successfully deleted
I20260925 09:30:37.136619 27216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:37.088320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58774 (local address 127.10.114.65:38917)
0925 09:30:37.088485 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:37.088492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:37.112429 (+ 23937us) server_negotiation.cc:407] Connection header received
0925 09:30:37.113640 (+ 1211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:37.113652 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:37.113742 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:37.113822 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:30:37.116904 (+ 3082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:37.117910 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:37.119974 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:37.120709 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:37.120821 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:37.129212 (+ 8391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:30:37.129253 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:30:37.129270 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:30:37.129312 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:30:37.133135 (+ 3823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:37.133859 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:37.133870 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:37.133872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:37.133990 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:37.134372 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:37.134376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:37.134378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:37.134799 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:30:37.135106 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:37.135450 (+ 344us) server_negotiation.cc:299] Negotiation successful
0925 09:30:37.136252 (+ 802us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 09:30:37.137286 26937 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:30:37.235761 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 26793
I20260925 09:30:37.257630 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 26946
I20260925 09:30:37.270709 27220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:37.258305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57744 (local address 127.10.114.66:33579)
0925 09:30:37.258463 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:37.258467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:37.258484 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:30:37.258640 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:37.258644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:37.258719 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:37.258808 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:30:37.260119 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:37.260800 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:37.262007 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:37.262369 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:37.262473 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:37.265132 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:30:37.265150 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:30:37.265165 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:30:37.265198 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:30:37.268372 (+ 3174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:37.268946 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:37.268953 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:37.268955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:37.269088 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:37.269418 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:37.269421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:37.269423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:37.269779 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:30:37.270004 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:37.270243 (+ 239us) server_negotiation.cc:299] Negotiation successful
0925 09:30:37.270408 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 09:30:37.271363 27069 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:30:37.361002 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 26946
I20260925 09:30:37.381170 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 27077
I20260925 09:30:37.394122 27234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:37.382041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44052 (local address 127.10.114.67:40219)
0925 09:30:37.382234 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 09:30:37.382239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:30:37.382283 (+ 44us) server_negotiation.cc:407] Connection header received
0925 09:30:37.382457 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:30:37.382460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:30:37.382549 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:30:37.382663 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:30:37.384258 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:37.384954 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:37.386178 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:30:37.386570 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:30:37.386673 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:30:37.389121 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:30:37.389134 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:30:37.389154 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:30:37.389192 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:30:37.391748 (+ 2556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:37.392402 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:37.392408 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:37.392410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:37.392532 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:30:37.392871 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:30:37.392873 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:30:37.392875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:30:37.393177 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:30:37.393392 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0925 09:30:37.393605 (+ 213us) server_negotiation.cc:299] Negotiation successful
0925 09:30:37.393777 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 09:30:37.394717 27200 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:30:37.480785 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 27077
I20260925 09:30:37.497611 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 26723
I20260925 09:30:37.498835 26786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:30:37.610661 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 26723
I20260925 09:30:57.632342 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:40439
--webserver_interface=127.10.114.126
--webserver_port=33937
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:40439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:30:57.947213 27256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:30:57.947476 27256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:30:57.947564 27256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:30:57.954710 27256 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 09:30:57.954799 27256 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 09:30:57.954864 27256 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 09:30:57.954908 27256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:30:57.954960 27256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:30:57.955026 27256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:30:57.955070 27256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:30:57.955116 27256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:30:57.964545 27256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:40439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:40439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=33937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:30:57.965632 27256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:30:57.966845 27256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:30:57.978037 27264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:57.978080 27262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:57.978124 27261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:30:57.979159 27256 server_base.cc:1034] running on GCE node
I20260925 09:30:57.979884 27256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:30:57.981071 27256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:30:57.982303 27256 hybrid_clock.cc:648] HybridClock initialized: now 1790328657982249 us; error 39 us; skew 500 ppm
I20260925 09:30:57.982622 27256 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:30:57 dist-test-slave-9pj8 krb5kdc[26170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328657, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:30:57.989568 27256 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:30:57.990967 27256 webserver.cc:460] Webserver started at http://127.10.114.126:33937/ using document root <none> and password file <none>
I20260925 09:30:57.991562 27256 fs_manager.cc:359] Metadata directory not provided
I20260925 09:30:57.991645 27256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:30:57.996816 27256 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:30:57.999526 27271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:58.000872 27256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:30:58.000998 27256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "a613cdc33a34404bbe16d80623c99f25"
format_stamp: "Formatted at 2026-09-25 09:30:11 on dist-test-slave-9pj8"
server_key: "3981ca57af12dd69e0328c8b4e2a56d9"
server_key_iv: "b4a130dbea1923bbda017dac5b3c516b"
server_key_version: "encryptionkey@0"
I20260925 09:30:58.001875 27256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:30:58.020705 27256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:30:58.024856 27256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:30:58.025100 27256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:30:58.043370 27256 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:40439
I20260925 09:30:58.043411 27322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:40439 every 8 connection(s)
I20260925 09:30:58.044783 27256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:30:58.045455 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 27256
I20260925 09:30:58.046384 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:38917
--local_ip_for_outbound_sockets=127.10.114.65
--tserver_master_addrs=127.10.114.126:40439
--webserver_port=39533
--webserver_interface=127.10.114.65
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:30:58.056864 27323 sys_catalog.cc:263] Verifying existing consensus state
I20260925 09:30:58.060102 27323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25: Bootstrap starting.
I20260925 09:30:58.080210 27323 log.cc:824] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25: Log is configured to *not* fsync() on all Append() calls
I20260925 09:30:58.095342 27323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 09:30:58.095940 27323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25: Bootstrap complete.
I20260925 09:30:58.104859 27323 raft_consensus.cc:348] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a613cdc33a34404bbe16d80623c99f25" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40439 } }
I20260925 09:30:58.105839 27323 raft_consensus.cc:729] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a613cdc33a34404bbe16d80623c99f25, State: Initialized, Role: FOLLOWER
I20260925 09:30:58.106468 27323 consensus_queue.cc:260] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a613cdc33a34404bbe16d80623c99f25" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40439 } }
I20260925 09:30:58.106719 27323 raft_consensus.cc:388] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:30:58.106841 27323 raft_consensus.cc:482] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:30:58.107013 27323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 2 FOLLOWER]: Advancing to term 3
I20260925 09:30:58.110632 27323 raft_consensus.cc:504] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a613cdc33a34404bbe16d80623c99f25" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40439 } }
I20260925 09:30:58.111111 27323 leader_election.cc:302] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a613cdc33a34404bbe16d80623c99f25; no voters:
I20260925 09:30:58.111845 27323 leader_election.cc:288] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 09:30:58.111971 27327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 09:30:58.113334 27327 raft_consensus.cc:686] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [term 3 LEADER]: Becoming Leader. State: Replica: a613cdc33a34404bbe16d80623c99f25, State: Running, Role: LEADER
I20260925 09:30:58.113833 27327 consensus_queue.cc:237] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a613cdc33a34404bbe16d80623c99f25" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40439 } }
I20260925 09:30:58.114068 27323 sys_catalog.cc:564] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:30:58.116525 27328 sys_catalog.cc:455] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a613cdc33a34404bbe16d80623c99f25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a613cdc33a34404bbe16d80623c99f25" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40439 } } }
I20260925 09:30:58.116917 27328 sys_catalog.cc:458] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [sys.catalog]: This master's current role is: LEADER
I20260925 09:30:58.116668 27329 sys_catalog.cc:455] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a613cdc33a34404bbe16d80623c99f25. Latest consensus state: current_term: 3 leader_uuid: "a613cdc33a34404bbe16d80623c99f25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a613cdc33a34404bbe16d80623c99f25" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40439 } } }
I20260925 09:30:58.117075 27329 sys_catalog.cc:458] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25 [sys.catalog]: This master's current role is: LEADER
I20260925 09:30:58.127555 27332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:30:58.136364 27332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:30:58.137766 27332 catalog_manager.cc:1259] Loaded cluster ID: 12ef670e78cc4c03b0e96a79717fef37
I20260925 09:30:58.137861 27332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:30:58.142148 27332 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:30:58.143651 27332 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25: Loaded TSK: 0
I20260925 09:30:58.143827 27332 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a613cdc33a34404bbe16d80623c99f25: Loaded TSK: 1
F20260925 09:30:58.144871 27332 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f72fbda09ad google::LogMessage::Fail() at ??:0
@ 0x7f72fbda388a google::LogMessage::SendToLog() at ??:0
@ 0x7f72fbda04de google::LogMessage::Flush() at ??:0
@ 0x7f72fbda1a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f72ffb047d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f72ffaf373e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f73117d31c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f73117d2783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f731185e2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f731185e0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f73118a85e9 std::function<>::operator()() at ??:0
@ 0x7f73117d6cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f73117d4c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f731185c610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f731185c439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f72fe9ec345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f72fe9ef91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f72fe9ef799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f72fe9caeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f7301ca46db start_thread at ??:0
@ 0x7f72fac3a61f clone at ??:0
W20260925 09:30:58.438978 27325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:30:58.439268 27325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:30:58.439370 27325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:30:58.446990 27325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:30:58.447105 27325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:30:58.447217 27325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:30:58.456547 27325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:38917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=39533
--tserver_master_addrs=127.10.114.126:40439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:30:58.457607 27325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:30:58.458787 27325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:30:58.471437 27370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:58.471472 27371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:58.471446 27373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:30:58.472551 27325 server_base.cc:1034] running on GCE node
I20260925 09:30:58.473166 27325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:30:58.473986 27325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:30:58.475178 27325 hybrid_clock.cc:648] HybridClock initialized: now 1790328658475134 us; error 30 us; skew 500 ppm
I20260925 09:30:58.475495 27325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:30:58 dist-test-slave-9pj8 krb5kdc[26170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328658, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:30:58.478458 27325 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:30:58.479877 27325 webserver.cc:460] Webserver started at http://127.10.114.65:39533/ using document root <none> and password file <none>
I20260925 09:30:58.480477 27325 fs_manager.cc:359] Metadata directory not provided
I20260925 09:30:58.480573 27325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:30:58.485524 27325 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260925 09:30:58.488034 27380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:58.489259 27325 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:30:58.489426 27325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "2a9f4e4a791044729f31d00e037e3845"
format_stamp: "Formatted at 2026-09-25 09:30:12 on dist-test-slave-9pj8"
server_key: "43fd67f0c87a53c63fa6a45f42b3f6cd"
server_key_iv: "e3d7fea6aac2ea4ea3ad896acd4c8c3c"
server_key_version: "encryptionkey@0"
I20260925 09:30:58.490360 27325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:30:58.528304 27325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:30:58.532815 27325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:30:58.533080 27325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:30:58.534016 27325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:30:58.535310 27325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:30:58.535414 27325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:58.535533 27325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:30:58.535630 27325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:58.569013 27325 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:38917
I20260925 09:30:58.569103 27492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:38917 every 8 connection(s)
I20260925 09:30:58.570533 27325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:30:58.570979 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 27325
I20260925 09:30:58.571731 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:33579
--local_ip_for_outbound_sockets=127.10.114.66
--tserver_master_addrs=127.10.114.126:40439
--webserver_port=37829
--webserver_interface=127.10.114.66
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:30:58.575966 27495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:58.573839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.65:36237)
0925 09:30:58.574697 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:58.575019 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":354,"threads_started":1}
W20260925 09:30:58.576650 27493 heartbeater.cc:644] Failed to heartbeat to 127.10.114.126:40439 (0 consecutive failures): Network error: Failed to ping master at 127.10.114.126:40439: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
I20260925 09:30:58.577489 27495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:58.577188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.65:52931)
0925 09:30:58.577325 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:58.577390 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260925 09:30:58.578277 27495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:58.577944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.65:35203)
0925 09:30:58.578095 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:58.578166 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 09:30:58.578485 27493 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 09:30:58.889565 27496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:30:58.889846 27496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:30:58.890005 27496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:30:58.897327 27496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:30:58.897419 27496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:30:58.897604 27496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:30:58.906956 27496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:33579
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=37829
--tserver_master_addrs=127.10.114.126:40439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:30:58.908186 27496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:30:58.909402 27496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:30:58.921787 27501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:58.921865 27502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:58.921980 27504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:30:58.922839 27496 server_base.cc:1034] running on GCE node
I20260925 09:30:58.923417 27496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:30:58.924378 27496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:30:58.925606 27496 hybrid_clock.cc:648] HybridClock initialized: now 1790328658925565 us; error 49 us; skew 500 ppm
I20260925 09:30:58.925902 27496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:30:58 dist-test-slave-9pj8 krb5kdc[26170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328658, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:30:58.929034 27496 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:30:58.930397 27496 webserver.cc:460] Webserver started at http://127.10.114.66:37829/ using document root <none> and password file <none>
I20260925 09:30:58.931018 27496 fs_manager.cc:359] Metadata directory not provided
I20260925 09:30:58.931156 27496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:30:58.936319 27496 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:30:58.938791 27511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:58.940194 27496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 09:30:58.940366 27496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "ab1d10dc32c3439f8dab35ca8c44afe2"
format_stamp: "Formatted at 2026-09-25 09:30:12 on dist-test-slave-9pj8"
server_key: "0e992f59ce9eb8f8da441b4bda81ecd3"
server_key_iv: "40557a36f9ea460098fdd6125cefef0e"
server_key_version: "encryptionkey@0"
I20260925 09:30:58.941300 27496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:30:58.956444 27496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:30:58.960471 27496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:30:58.960729 27496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:30:58.961608 27496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:30:58.962873 27496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:30:58.962992 27496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:58.963106 27496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:30:58.963193 27496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:58.999081 27496 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:33579
I20260925 09:30:58.999133 27623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:33579 every 8 connection(s)
I20260925 09:30:59.000651 27496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:30:59.004577 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 27496
I20260925 09:30:59.005199 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:40219
--local_ip_for_outbound_sockets=127.10.114.67
--tserver_master_addrs=127.10.114.126:40439
--webserver_port=39227
--webserver_interface=127.10.114.67
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:30:59.005820 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:59.003581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.66:37649)
0925 09:30:59.004380 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:59.004692 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":307,"threads_started":1}
W20260925 09:30:59.006593 27624 heartbeater.cc:644] Failed to heartbeat to 127.10.114.126:40439 (0 consecutive failures): Network error: Failed to ping master at 127.10.114.126:40439: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
I20260925 09:30:59.007571 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:59.007210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.66:46001)
0925 09:30:59.007383 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:59.007458 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260925 09:30:59.008590 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:59.008213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.66:45077)
0925 09:30:59.008377 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:59.008468 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260925 09:30:59.008895 27624 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 09:30:59.320392 27627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:30:59.320616 27627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:30:59.320809 27627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:30:59.327605 27627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:30:59.327698 27627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:30:59.327880 27627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:30:59.337493 27627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:40219
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=39227
--tserver_master_addrs=127.10.114.126:40439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:30:59.338639 27627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:30:59.340054 27627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:30:59.352887 27633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:59.352926 27635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:30:59.352977 27632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:30:59.353718 27627 server_base.cc:1034] running on GCE node
I20260925 09:30:59.354270 27627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:30:59.355196 27627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:30:59.356400 27627 hybrid_clock.cc:648] HybridClock initialized: now 1790328659356344 us; error 40 us; skew 500 ppm
I20260925 09:30:59.356685 27627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:30:59 dist-test-slave-9pj8 krb5kdc[26170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328659, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:30:59.359639 27627 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:30:59.360965 27627 webserver.cc:460] Webserver started at http://127.10.114.67:39227/ using document root <none> and password file <none>
I20260925 09:30:59.361613 27627 fs_manager.cc:359] Metadata directory not provided
I20260925 09:30:59.361776 27627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:30:59.366731 27627 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.001s
I20260925 09:30:59.369372 27642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:59.370618 27627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 09:30:59.370786 27627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "d8fea1757f77441b9115d8680c4f085d"
format_stamp: "Formatted at 2026-09-25 09:30:13 on dist-test-slave-9pj8"
server_key: "d3e50694b23a184bbf5096de9ddb0d94"
server_key_iv: "a6fd0ef9722b278a3be7b529789dac4f"
server_key_version: "encryptionkey@0"
I20260925 09:30:59.371744 27627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:30:59.402686 27627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:30:59.406759 27627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:30:59.407030 27627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:30:59.408053 27627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:30:59.409292 27627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:30:59.409405 27627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:59.409533 27627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:30:59.409624 27627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:30:59.442981 27627 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:40219
I20260925 09:30:59.443068 27754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:40219 every 8 connection(s)
I20260925 09:30:59.444473 27627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:30:59.448915 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 27627
I20260925 09:30:59.449503 27757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:59.447558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.67:38131)
0925 09:30:59.448314 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:59.448551 (+ 237us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":339,"threads_started":1}
W20260925 09:30:59.450290 27755 heartbeater.cc:644] Failed to heartbeat to 127.10.114.126:40439 (0 consecutive failures): Network error: Failed to ping master at 127.10.114.126:40439: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
I20260925 09:30:59.451116 27757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:59.450811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.67:39033)
0925 09:30:59.450952 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:59.451021 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260925 09:30:59.451947 27757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:59.451619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.67:33251)
0925 09:30:59.451740 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:59.451849 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260925 09:30:59.452180 27755 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 09:30:59.579828 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:30:59.579028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.65:34263)
0925 09:30:59.579495 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0925 09:30:59.579577 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":248,"threads_started":1}
I20260925 09:31:00.010195 27760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:00.009398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.66:35869)
0925 09:31:00.009842 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:00.009949 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":227,"threads_started":1}
I20260925 09:31:00.453347 27761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:00.452649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.67:47869)
0925 09:31:00.453054 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:00.453134 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":201,"threads_started":1}
I20260925 09:31:00.581357 27762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:00.580648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.65:50277)
0925 09:31:00.581102 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:00.581191 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":220,"threads_started":1}
I20260925 09:31:01.011703 27763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:01.011015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.66:47613)
0925 09:31:01.011456 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:01.011533 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":225,"threads_started":1}
I20260925 09:31:01.454882 27764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:01.454250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.67:40905)
0925 09:31:01.454656 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:01.454731 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":191,"threads_started":1}
I20260925 09:31:01.583067 27765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:01.582361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.65:53929)
0925 09:31:01.582818 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:01.582897 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":260,"threads_started":1}
I20260925 09:31:02.013304 27766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:02.012583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.66:52351)
0925 09:31:02.012993 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:02.013123 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":228,"threads_started":1}
I20260925 09:31:02.456338 27767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:02.455629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.67:46757)
0925 09:31:02.456104 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:02.456183 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":287,"threads_started":1}
I20260925 09:31:02.584764 27768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:02.583938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.65:34351)
0925 09:31:02.584498 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:02.584579 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":326,"threads_started":1}
I20260925 09:31:03.014899 27769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:03.014126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.66:49561)
0925 09:31:03.014621 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:03.014698 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":288,"threads_started":1}
I20260925 09:31:03.457844 27770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:03.457145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.67:55289)
0925 09:31:03.457596 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:03.457671 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":242,"threads_started":1}
I20260925 09:31:03.586424 27771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:03.585720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.65:52553)
0925 09:31:03.586194 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:03.586269 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":220,"threads_started":1}
I20260925 09:31:04.016592 27772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:04.015850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.66:51073)
0925 09:31:04.016330 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:04.016415 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":247,"threads_started":1}
I20260925 09:31:04.459364 27773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:04.458695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.67:46211)
0925 09:31:04.459134 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:04.459212 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":240,"threads_started":1}
I20260925 09:31:04.588040 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:04.587274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.65:57341)
0925 09:31:04.587739 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:04.587887 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":254,"threads_started":1}
I20260925 09:31:05.018220 27775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:05.017490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.66:59675)
0925 09:31:05.017977 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:05.018062 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":283,"threads_started":1}
I20260925 09:31:05.460975 27776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:05.460306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.67:41527)
0925 09:31:05.460711 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:05.460827 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":210,"threads_started":1}
I20260925 09:31:05.589826 27777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:05.588986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.65:46689)
0925 09:31:05.589531 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:05.589616 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":257,"threads_started":1}
I20260925 09:31:06.020118 27778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:06.019123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.66:44193)
0925 09:31:06.019712 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:06.019852 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":293,"threads_started":1}
I20260925 09:31:06.462803 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:06.461932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.67:57567)
0925 09:31:06.462524 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:06.462615 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":297,"threads_started":1}
I20260925 09:31:06.591413 27780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:06.590760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.65:37921)
0925 09:31:06.591173 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:06.591256 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":222,"threads_started":1}
I20260925 09:31:07.021749 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:07.021030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.66:49039)
0925 09:31:07.021491 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:07.021571 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":253,"threads_started":1}
I20260925 09:31:07.464453 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:07.463672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.67:46687)
0925 09:31:07.464193 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:07.464298 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":310,"threads_started":1}
I20260925 09:31:07.593256 27783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:07.592401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.65:43241)
0925 09:31:07.592881 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:07.592988 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":241,"threads_started":1}
I20260925 09:31:08.023607 27784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:08.022775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.66:45591)
0925 09:31:08.023327 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:08.023420 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":280,"threads_started":1}
I20260925 09:31:08.466005 27785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:08.465305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.67:42845)
0925 09:31:08.465747 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:08.465849 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":214,"threads_started":1}
I20260925 09:31:08.594754 27786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:08.594142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.65:54891)
0925 09:31:08.594535 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:08.594610 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":220,"threads_started":1}
I20260925 09:31:09.025393 27787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:09.024679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.66:35461)
0925 09:31:09.025133 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:09.025235 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":251,"threads_started":1}
I20260925 09:31:09.467491 27788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:09.466849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.67:40607)
0925 09:31:09.467269 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:09.467345 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":224,"threads_started":1}
I20260925 09:31:09.596408 27789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:09.595715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.65:45229)
0925 09:31:09.596155 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:09.596243 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":239,"threads_started":1}
I20260925 09:31:10.027030 27790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:10.026297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.66:51107)
0925 09:31:10.026760 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:10.026852 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":230,"threads_started":1}
I20260925 09:31:10.469041 27791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:10.468400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.67:43839)
0925 09:31:10.468799 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:10.468880 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":208,"threads_started":1}
I20260925 09:31:10.597914 27792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:10.597281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.65:40295)
0925 09:31:10.597675 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:10.597765 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":203,"threads_started":1}
I20260925 09:31:11.028736 27793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:11.028052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.66:49189)
0925 09:31:11.028509 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:11.028587 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":251,"threads_started":1}
I20260925 09:31:11.470603 27794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:11.469946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.67:39357)
0925 09:31:11.470367 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:11.470450 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":218,"threads_started":1}
I20260925 09:31:11.599628 27795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:11.598900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.65:56875)
0925 09:31:11.599359 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:11.599452 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":226,"threads_started":1}
I20260925 09:31:12.030490 27796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:12.029704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.66:52213)
0925 09:31:12.030181 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:12.030303 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":233,"threads_started":1}
I20260925 09:31:12.472388 27797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:12.471472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.67:45935)
0925 09:31:12.472054 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:12.472157 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":260,"threads_started":1}
I20260925 09:31:12.601275 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:12.600551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.65:47821)
0925 09:31:12.601005 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:12.601089 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":244,"threads_started":1}
I20260925 09:31:13.032210 27799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:13.031447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.66:47201)
0925 09:31:13.031938 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:13.032029 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":211,"threads_started":1}
I20260925 09:31:13.473922 27800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:13.473261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.67:38655)
0925 09:31:13.473650 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:13.473752 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":200,"threads_started":1}
I20260925 09:31:13.603068 27801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:13.602349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.65:55327)
0925 09:31:13.602789 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:13.602886 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":220,"threads_started":1}
I20260925 09:31:14.034073 27802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:14.033338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.66:35729)
0925 09:31:14.033793 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:14.033891 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":227,"threads_started":1}
I20260925 09:31:14.454731 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 27325
I20260925 09:31:14.471735 27803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:14.455680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46490 (local address 127.10.114.65:38917)
0925 09:31:14.456357 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:14.456374 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:14.456442 (+ 68us) server_negotiation.cc:407] Connection header received
0925 09:31:14.458076 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:14.458085 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:14.458503 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:14.458825 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:14.460220 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:14.461387 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:14.462599 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:14.463207 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:14.463336 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:14.466037 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:14.466053 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:14.466066 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:14.466107 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:14.468810 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:14.469499 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:14.469505 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:14.469510 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:14.469658 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:14.469997 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:14.469999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:14.470001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:14.470450 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:14.470680 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:14.471076 (+ 396us) server_negotiation.cc:299] Negotiation successful
0925 09:31:14.471476 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":358,"threads_started":1}
I20260925 09:31:14.473228 27488 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:14.475957 27805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:14.474913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40439 (local address 127.10.114.67:39111)
0925 09:31:14.475452 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:14.475610 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.126:40439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":234,"threads_started":1}
I20260925 09:31:14.563485 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 27325
I20260925 09:31:14.580413 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 27496
I20260925 09:31:14.597774 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:14.581364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52976 (local address 127.10.114.66:33579)
0925 09:31:14.581896 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:14.581910 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:14.581948 (+ 38us) server_negotiation.cc:407] Connection header received
0925 09:31:14.583487 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:14.583496 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:14.583989 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:14.584351 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:14.585767 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:14.587025 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:14.588292 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:14.588951 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:14.589164 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:14.591644 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:14.591652 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:14.591666 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:14.591699 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:14.594383 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:14.595047 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:14.595059 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:14.595065 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:14.595227 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:14.595607 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:14.595612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:14.595615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:14.596307 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:14.596634 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:14.597100 (+ 466us) server_negotiation.cc:299] Negotiation successful
0925 09:31:14.597494 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":257,"threads_started":1}
I20260925 09:31:14.599126 27619 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:14.681689 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 27496
I20260925 09:31:14.700589 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 27627
I20260925 09:31:14.717340 27808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:14.701636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44148 (local address 127.10.114.67:40219)
0925 09:31:14.702259 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:14.702273 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:14.702314 (+ 41us) server_negotiation.cc:407] Connection header received
0925 09:31:14.704006 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:14.704017 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:14.704481 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:14.704743 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:14.705997 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:14.707234 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:14.708466 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:14.709112 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:14.709244 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:14.711731 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:14.711738 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:14.711750 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:14.711827 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:14.714625 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:14.715267 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:14.715272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:14.715278 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:14.715431 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:14.715736 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:14.715739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:14.715741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:14.716256 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:14.716465 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:14.716772 (+ 307us) server_negotiation.cc:299] Negotiation successful
0925 09:31:14.717074 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":311,"threads_started":1}
I20260925 09:31:14.718648 27750 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:14.798059 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 27627
2026-09-25T09:31:14Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64689 ms)
[----------] 30 tests from SecurityITest (289463 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:31:14 dist-test-slave-9pj8 krb5kdc[27812](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:31:14 dist-test-slave-9pj8 krb5kdc[27812](info): set up 2 sockets
Sep 25 09:31:14 dist-test-slave-9pj8 krb5kdc[27812](info): commencing operation
krb5kdc: starting...
W20260925 09:31:15.892364 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:31:16 dist-test-slave-9pj8 krb5kdc[27812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:31:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:31:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:31:16.122633 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:40829
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:40829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:16.435376 27825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:16.435683 27825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:16.435863 27825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:16.442806 27825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:31:16.442934 27825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:16.443034 27825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:16.443132 27825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:31:16.443200 27825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:31:16.452574 27825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:40829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:40829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:16.453815 27825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:16.455091 27825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:16.466748 27833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:16.466809 27830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:16.466785 27831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:16.467533 27825 server_base.cc:1034] running on GCE node
I20260925 09:31:16.468356 27825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:16.469650 27825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:16.470873 27825 hybrid_clock.cc:648] HybridClock initialized: now 1790328676470831 us; error 36 us; skew 500 ppm
I20260925 09:31:16.471196 27825 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:16 dist-test-slave-9pj8 krb5kdc[27812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328676, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:16.474182 27825 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:31:16.475616 27825 webserver.cc:460] Webserver started at http://127.10.114.126:41095/ using document root <none> and password file <none>
I20260925 09:31:16.476285 27825 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:16.476421 27825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:16.476698 27825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:16.479549 27825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "947f42665232458dbc8ea44232a1248e"
format_stamp: "Formatted at 2026-09-25 09:31:16 on dist-test-slave-9pj8"
server_key: "155452054181d7a0ed7a0706c55d5726"
server_key_iv: "c981df3fc865e5ed6e676ee3faa9828a"
server_key_version: "encryptionkey@0"
I20260925 09:31:16.480299 27825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "947f42665232458dbc8ea44232a1248e"
format_stamp: "Formatted at 2026-09-25 09:31:16 on dist-test-slave-9pj8"
server_key: "155452054181d7a0ed7a0706c55d5726"
server_key_iv: "c981df3fc865e5ed6e676ee3faa9828a"
server_key_version: "encryptionkey@0"
I20260925 09:31:16.485107 27825 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260925 09:31:16.488329 27840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:16.489506 27825 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 09:31:16.489668 27825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "947f42665232458dbc8ea44232a1248e"
format_stamp: "Formatted at 2026-09-25 09:31:16 on dist-test-slave-9pj8"
server_key: "155452054181d7a0ed7a0706c55d5726"
server_key_iv: "c981df3fc865e5ed6e676ee3faa9828a"
server_key_version: "encryptionkey@0"
I20260925 09:31:16.489846 27825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:16.504940 27825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:16.508747 27825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:16.509014 27825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:16.526721 27825 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:40829
I20260925 09:31:16.526762 27891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:40829 every 8 connection(s)
I20260925 09:31:16.528101 27825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:31:16.531574 27892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:16.533468 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 27825
I20260925 09:31:16.533690 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:31:16.534480 10697 external_mini_cluster.cc:1428] Setting key 3f7e782f6babfd8ac7502d2cef777d0c
I20260925 09:31:16.545599 27892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e: Bootstrap starting.
Sep 25 09:31:16 dist-test-slave-9pj8 krb5kdc[27812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:31:16.549381 27892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:16.550398 27892 log.cc:824] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:16.553213 27892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e: No bootstrap required, opened a new log
I20260925 09:31:16.557366 27895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:16.536445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45572 (local address 127.10.114.126:40829)
0925 09:31:16.537221 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:16.537232 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:16.537273 (+ 41us) server_negotiation.cc:407] Connection header received
0925 09:31:16.538760 (+ 1487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:16.538782 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:16.539178 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:16.539476 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:16.541238 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:16.542496 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:16.544422 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:16.545023 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:16.545141 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:16.549954 (+ 4813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:16.549962 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:16.549975 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:16.550008 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:16.553491 (+ 3483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:16.554164 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:16.554171 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:16.554179 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:16.554350 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:16.554714 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:16.554718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:16.554722 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:16.555222 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:16.555542 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:16.555991 (+ 449us) server_negotiation.cc:299] Negotiation successful
0925 09:31:16.556449 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":355,"threads_started":1}
I20260925 09:31:16.560720 27892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "947f42665232458dbc8ea44232a1248e" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40829 } }
I20260925 09:31:16.561066 27892 raft_consensus.cc:374] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:16.561174 27892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947f42665232458dbc8ea44232a1248e, State: Initialized, Role: FOLLOWER
I20260925 09:31:16.561846 27892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "947f42665232458dbc8ea44232a1248e" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40829 } }
I20260925 09:31:16.562147 27892 raft_consensus.cc:388] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:31:16.562285 27892 raft_consensus.cc:482] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:31:16.562438 27892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:16.564880 27892 raft_consensus.cc:504] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "947f42665232458dbc8ea44232a1248e" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40829 } }
I20260925 09:31:16.565408 27892 leader_election.cc:302] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 947f42665232458dbc8ea44232a1248e; no voters:
I20260925 09:31:16.566087 27892 leader_election.cc:288] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:31:16.566196 27897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:16.566797 27897 raft_consensus.cc:686] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e [term 1 LEADER]: Becoming Leader. State: Replica: 947f42665232458dbc8ea44232a1248e, State: Running, Role: LEADER
I20260925 09:31:16.567351 27897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "947f42665232458dbc8ea44232a1248e" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40829 } }
I20260925 09:31:16.568109 27892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:31:16.570206 27898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "947f42665232458dbc8ea44232a1248e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "947f42665232458dbc8ea44232a1248e" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40829 } } }
I20260925 09:31:16.571511 27898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:16.570990 27899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 947f42665232458dbc8ea44232a1248e. Latest consensus state: current_term: 1 leader_uuid: "947f42665232458dbc8ea44232a1248e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "947f42665232458dbc8ea44232a1248e" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40829 } } }
I20260925 09:31:16.574271 27899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:16.574323 27905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:31:16.579308 27905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:31:16.590626 27905 catalog_manager.cc:1347] Generated new cluster ID: a42011225bc44e5aae07632fd5b95d05
I20260925 09:31:16.590723 27905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:31:16.613466 27905 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:31:16.615204 27905 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:31:16.626911 27905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e: Generated new TSK 0
I20260925 09:31:16.627691 27905 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
I20260925 09:31:16.689623 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:40829
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:17.011250 27918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:17.011602 27918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:17.011786 27918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:17.018981 27918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:17.019106 27918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:17.019280 27918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:31:17.028882 27918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:40829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:17.030136 27918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:17.031472 27918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:17.044399 27923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:17.044374 27924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:17.044606 27926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:17.045501 27918 server_base.cc:1034] running on GCE node
I20260925 09:31:17.046030 27918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:17.046871 27918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:17.048074 27918 hybrid_clock.cc:648] HybridClock initialized: now 1790328677048023 us; error 44 us; skew 500 ppm
I20260925 09:31:17.048372 27918 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:17 dist-test-slave-9pj8 krb5kdc[27812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328677, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:17.055127 27918 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:31:17.056497 27918 webserver.cc:460] Webserver started at http://127.10.114.65:41833/ using document root <none> and password file <none>
I20260925 09:31:17.057215 27918 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:17.057312 27918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:17.057562 27918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:17.060433 27918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "c1c0b1afb5244e2684c04c28e1ab2b9b"
format_stamp: "Formatted at 2026-09-25 09:31:17 on dist-test-slave-9pj8"
server_key: "ba544747ea3d722896b8cd064e02b8eb"
server_key_iv: "7442d2291bccd3d2a466c43e49579a53"
server_key_version: "encryptionkey@0"
I20260925 09:31:17.061066 27918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "c1c0b1afb5244e2684c04c28e1ab2b9b"
format_stamp: "Formatted at 2026-09-25 09:31:17 on dist-test-slave-9pj8"
server_key: "ba544747ea3d722896b8cd064e02b8eb"
server_key_iv: "7442d2291bccd3d2a466c43e49579a53"
server_key_version: "encryptionkey@0"
I20260925 09:31:17.065795 27918 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:31:17.068991 27933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:17.070410 27918 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:31:17.070559 27918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "c1c0b1afb5244e2684c04c28e1ab2b9b"
format_stamp: "Formatted at 2026-09-25 09:31:17 on dist-test-slave-9pj8"
server_key: "ba544747ea3d722896b8cd064e02b8eb"
server_key_iv: "7442d2291bccd3d2a466c43e49579a53"
server_key_version: "encryptionkey@0"
I20260925 09:31:17.070715 27918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:17.103415 27918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:17.107653 27918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:17.107949 27918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:17.108862 27918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:31:17.110162 27918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:31:17.110231 27918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:17.110365 27918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:31:17.110414 27918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:17.144349 27918 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:39587
I20260925 09:31:17.144387 28045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:39587 every 8 connection(s)
I20260925 09:31:17.145704 27918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:31:17.150986 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 27918
I20260925 09:31:17.151221 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:31:17.151948 10697 external_mini_cluster.cc:1428] Setting key 907e6d6dc0175802bc92e72c642892c1
Sep 25 09:31:17 dist-test-slave-9pj8 krb5kdc[27812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328677, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:31:17.166021 28048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:17.148276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:36597 (local address 127.10.114.126:40829)
0925 09:31:17.148738 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:17.148742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:17.149653 (+ 911us) server_negotiation.cc:407] Connection header received
0925 09:31:17.151501 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:17.151507 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:17.151593 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:17.151707 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:17.153409 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:17.154276 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:17.155823 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:17.156219 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:17.156366 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:17.159983 (+ 3617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:17.159988 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:17.159991 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:17.160023 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:17.163047 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:17.163714 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:17.163722 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:17.163727 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:17.163916 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:17.164355 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:17.164361 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:17.164364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:17.164615 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:17.164764 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:17.165453 (+ 689us) server_negotiation.cc:299] Negotiation successful
0925 09:31:17.165666 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":258,"threads_started":1}
I20260925 09:31:17.166953 28049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:17.148805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40829 (local address 127.10.114.65:36597)
0925 09:31:17.149492 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:17.149529 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:17.151213 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:17.151966 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:17.151973 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:17.152542 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:31:17.153224 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:17.153242 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:17.154448 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:17.154453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:17.155671 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:17.155682 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:17.156440 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:17.156449 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:17.156684 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:17.157370 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:31:17.157395 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:31:17.159673 (+ 2278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:31:17.163181 (+ 3508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:31:17.163189 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:31:17.163205 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:31:17.163598 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:31:17.164045 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:31:17.164050 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:31:17.164054 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:31:17.164234 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:31:17.164751 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:31:17.164759 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:31:17.165083 (+ 324us) client_negotiation.cc:769] Sending connection context
0925 09:31:17.165344 (+ 261us) client_negotiation.cc:240] Negotiation successful
0925 09:31:17.165838 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":286,"threads_started":1}
I20260925 09:31:17.168648 28046 heartbeater.cc:347] Connected to a master server at 127.10.114.126:40829
I20260925 09:31:17.168982 28046 heartbeater.cc:464] Registering TS with master...
I20260925 09:31:17.169703 28046 heartbeater.cc:511] Master 127.10.114.126:40829 requested a full tablet report, sending...
I20260925 09:31:17.171967 27857 ts_manager.cc:194] Registered new tserver with Master: c1c0b1afb5244e2684c04c28e1ab2b9b (127.10.114.65:39587)
I20260925 09:31:17.174109 27857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:36597
WARNING: no policy specified for kudu/127.10.114.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
I20260925 09:31:17.206542 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:40829
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:17.518815 28052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:17.519135 28052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:17.519318 28052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:17.526467 28052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:17.526604 28052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:17.526769 28052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:31:17.536080 28052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:40829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:17.537246 28052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:17.538446 28052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:17.551321 28058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:17.551386 28057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:17.551403 28060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:17.552350 28052 server_base.cc:1034] running on GCE node
I20260925 09:31:17.552829 28052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:17.553618 28052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:17.554800 28052 hybrid_clock.cc:648] HybridClock initialized: now 1790328677554748 us; error 45 us; skew 500 ppm
I20260925 09:31:17.555086 28052 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:17 dist-test-slave-9pj8 krb5kdc[27812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328677, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:17.558035 28052 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:31:17.559449 28052 webserver.cc:460] Webserver started at http://127.10.114.66:34723/ using document root <none> and password file <none>
I20260925 09:31:17.560141 28052 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:17.560281 28052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:17.560561 28052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:17.563187 28052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "157169846f494903bab802b8a6f8f7cd"
format_stamp: "Formatted at 2026-09-25 09:31:17 on dist-test-slave-9pj8"
server_key: "7133b6db7b866881025a80ae87f88e56"
server_key_iv: "9242a49140783218945e84e715954db7"
server_key_version: "encryptionkey@0"
I20260925 09:31:17.563994 28052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "157169846f494903bab802b8a6f8f7cd"
format_stamp: "Formatted at 2026-09-25 09:31:17 on dist-test-slave-9pj8"
server_key: "7133b6db7b866881025a80ae87f88e56"
server_key_iv: "9242a49140783218945e84e715954db7"
server_key_version: "encryptionkey@0"
I20260925 09:31:17.568845 28052 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 09:31:17.572103 28067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:17.573504 28052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 09:31:17.573658 28052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "157169846f494903bab802b8a6f8f7cd"
format_stamp: "Formatted at 2026-09-25 09:31:17 on dist-test-slave-9pj8"
server_key: "7133b6db7b866881025a80ae87f88e56"
server_key_iv: "9242a49140783218945e84e715954db7"
server_key_version: "encryptionkey@0"
I20260925 09:31:17.573854 28052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:17.585469 28052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:17.589417 28052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:17.589679 28052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:17.590519 28052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:31:17.591702 28052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:31:17.591816 28052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:17.591953 28052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:31:17.592015 28052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:17.623389 28052 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:39853
I20260925 09:31:17.623431 28179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:39853 every 8 connection(s)
I20260925 09:31:17.624689 28052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:31:17.627264 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 28052
I20260925 09:31:17.627449 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:31:17.627948 10697 external_mini_cluster.cc:1428] Setting key 5b199cf151ac42ab2870aa84add2a47c
Sep 25 09:31:17 dist-test-slave-9pj8 krb5kdc[27812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328677, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:31:17.645934 28048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:17.627394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:46211 (local address 127.10.114.126:40829)
0925 09:31:17.627580 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:17.627587 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:17.629079 (+ 1492us) server_negotiation.cc:407] Connection header received
0925 09:31:17.630980 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:17.630988 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:17.631076 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:17.631187 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:17.632762 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:17.633859 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:17.635560 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:17.636358 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:17.636513 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:17.640350 (+ 3837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:17.640355 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:17.640358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:17.640390 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:17.642977 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:17.643678 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:17.643688 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:17.643691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:17.643868 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:17.644292 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:17.644297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:17.644300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:17.644548 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:17.644701 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:17.645364 (+ 663us) server_negotiation.cc:299] Negotiation successful
0925 09:31:17.645581 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 09:31:17.646848 28182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:17.628014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40829 (local address 127.10.114.66:46211)
0925 09:31:17.628914 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:17.628963 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:17.630704 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:17.631391 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:17.631398 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:17.632007 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:31:17.632585 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:17.632603 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:17.634040 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:17.634045 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:17.635406 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:17.635419 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:17.636599 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:17.636609 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:17.636925 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:17.637757 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:31:17.637781 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:31:17.640132 (+ 2351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:31:17.643126 (+ 2994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:31:17.643135 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:31:17.643151 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:31:17.643551 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:31:17.643984 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:31:17.643989 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:31:17.643993 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:31:17.644169 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:31:17.644673 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:31:17.644681 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:31:17.645005 (+ 324us) client_negotiation.cc:769] Sending connection context
0925 09:31:17.645248 (+ 243us) client_negotiation.cc:240] Negotiation successful
0925 09:31:17.645736 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":373,"threads_started":1}
I20260925 09:31:17.648533 28180 heartbeater.cc:347] Connected to a master server at 127.10.114.126:40829
I20260925 09:31:17.648833 28180 heartbeater.cc:464] Registering TS with master...
I20260925 09:31:17.649533 28180 heartbeater.cc:511] Master 127.10.114.126:40829 requested a full tablet report, sending...
I20260925 09:31:17.650980 27857 ts_manager.cc:194] Registered new tserver with Master: 157169846f494903bab802b8a6f8f7cd (127.10.114.66:39853)
I20260925 09:31:17.651814 27857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:46211
WARNING: no policy specified for kudu/127.10.114.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
I20260925 09:31:17.684727 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:40829
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:17.995221 28185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:17.995504 28185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:17.995666 28185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:18.002542 28185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:18.002679 28185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:18.002866 28185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:31:18.012048 28185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:40829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:18.013322 28185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:18.014653 28185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:18.026932 28190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:18.027020 28191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:18.027158 28193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:18.027962 28185 server_base.cc:1034] running on GCE node
I20260925 09:31:18.028568 28185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:18.029457 28185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:18.030697 28185 hybrid_clock.cc:648] HybridClock initialized: now 1790328678030651 us; error 56 us; skew 500 ppm
I20260925 09:31:18.031000 28185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:18 dist-test-slave-9pj8 krb5kdc[27812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328678, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:18.034533 28185 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:31:18.035971 28185 webserver.cc:460] Webserver started at http://127.10.114.67:33925/ using document root <none> and password file <none>
I20260925 09:31:18.036623 28185 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:18.036767 28185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:18.037075 28185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:18.039753 28185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "40c4e615d99a48b592818053114667b1"
format_stamp: "Formatted at 2026-09-25 09:31:18 on dist-test-slave-9pj8"
server_key: "b2823a3919b2c44b36a17250d35f7351"
server_key_iv: "a14068d912b77e414ef952759e2434ab"
server_key_version: "encryptionkey@0"
I20260925 09:31:18.040519 28185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "40c4e615d99a48b592818053114667b1"
format_stamp: "Formatted at 2026-09-25 09:31:18 on dist-test-slave-9pj8"
server_key: "b2823a3919b2c44b36a17250d35f7351"
server_key_iv: "a14068d912b77e414ef952759e2434ab"
server_key_version: "encryptionkey@0"
I20260925 09:31:18.045162 28185 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:31:18.048318 28200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:18.049531 28185 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:31:18.049688 28185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "40c4e615d99a48b592818053114667b1"
format_stamp: "Formatted at 2026-09-25 09:31:18 on dist-test-slave-9pj8"
server_key: "b2823a3919b2c44b36a17250d35f7351"
server_key_iv: "a14068d912b77e414ef952759e2434ab"
server_key_version: "encryptionkey@0"
I20260925 09:31:18.049876 28185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:18.084167 28185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:18.088001 28185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:18.088275 28185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:18.089188 28185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:31:18.090448 28185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:31:18.090546 28185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:18.090685 28185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:31:18.090752 28185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:18.124038 28185 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:44533
I20260925 09:31:18.124120 28312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:44533 every 8 connection(s)
I20260925 09:31:18.125308 28185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:31:18.135486 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 28185
I20260925 09:31:18.135679 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:31:18.136152 10697 external_mini_cluster.cc:1428] Setting key 98a810133398ee611c8b587af975597b
Sep 25 09:31:18 dist-test-slave-9pj8 krb5kdc[27812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328678, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:31:18.144771 28048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:18.127884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:51749 (local address 127.10.114.126:40829)
0925 09:31:18.128058 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:18.128062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:18.129122 (+ 1060us) server_negotiation.cc:407] Connection header received
0925 09:31:18.130942 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:18.130948 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:18.131040 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:18.131152 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:18.132621 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.133434 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.134894 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.135521 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.135659 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:18.139437 (+ 3778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:18.139442 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:18.139445 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:18.139474 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:18.141893 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:18.142558 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:18.142562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:18.142565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:18.142710 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:18.143122 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:18.143125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:18.143127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:18.143391 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:18.143502 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:18.144371 (+ 869us) server_negotiation.cc:299] Negotiation successful
0925 09:31:18.144525 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:31:18.145967 28315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:18.128336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40829 (local address 127.10.114.67:51749)
0925 09:31:18.128979 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:18.129015 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:18.130681 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:18.131382 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:18.131390 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:18.131921 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:31:18.132457 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:18.132474 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.133586 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.133590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:18.134755 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:18.134766 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.135822 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.135831 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:18.136041 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:18.136790 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:31:18.136827 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:31:18.139261 (+ 2434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:31:18.142006 (+ 2745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:31:18.142011 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:31:18.142021 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:31:18.142415 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:31:18.142826 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:31:18.142830 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:31:18.142832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:31:18.143000 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:31:18.143522 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:31:18.143530 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:31:18.143939 (+ 409us) client_negotiation.cc:769] Sending connection context
0925 09:31:18.144218 (+ 279us) client_negotiation.cc:240] Negotiation successful
0925 09:31:18.144686 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":252,"threads_started":1}
I20260925 09:31:18.147324 28313 heartbeater.cc:347] Connected to a master server at 127.10.114.126:40829
I20260925 09:31:18.147617 28313 heartbeater.cc:464] Registering TS with master...
I20260925 09:31:18.148334 28313 heartbeater.cc:511] Master 127.10.114.126:40829 requested a full tablet report, sending...
I20260925 09:31:18.149827 27857 ts_manager.cc:194] Registered new tserver with Master: 40c4e615d99a48b592818053114667b1 (127.10.114.67:44533)
I20260925 09:31:18.150712 27857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:51749
I20260925 09:31:18.151178 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:31:18.168262 28048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:18.156259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45578 (local address 127.10.114.126:40829)
0925 09:31:18.156397 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:18.156401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:18.156897 (+ 496us) server_negotiation.cc:407] Connection header received
0925 09:31:18.157050 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:18.157054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:18.157109 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:18.157186 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:18.158550 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.159268 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.160877 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.161265 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.161355 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:18.163602 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:18.163606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:18.163609 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:18.163646 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:18.166015 (+ 2369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:18.166611 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:18.166614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:18.166616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:18.166778 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:18.167092 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:18.167095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:18.167097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:18.167372 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:18.167541 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:18.167888 (+ 347us) server_negotiation.cc:299] Negotiation successful
0925 09:31:18.168038 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 09:31:18.177389 28046 heartbeater.cc:503] Master 127.10.114.126:40829 was elected leader, sending a full tablet report...
I20260925 09:31:18.172190 27857 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45578:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:31:18.179992 27857 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:31:18.210011 28328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:18.193204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47886 (local address 127.10.114.66:39853)
0925 09:31:18.193768 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:18.193776 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:18.201090 (+ 7314us) server_negotiation.cc:407] Connection header received
0925 09:31:18.201400 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:18.201412 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:18.201727 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:18.201941 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:18.203402 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.204866 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.207526 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.208726 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.208866 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:18.209079 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:18.209418 (+ 339us) server_negotiation.cc:299] Negotiation successful
0925 09:31:18.209595 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":827,"threads_started":1}
I20260925 09:31:18.210781 28329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:18.191943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:39853 (local address 127.0.0.1:47886)
0925 09:31:18.200510 (+ 8567us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:18.200533 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:18.200690 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:18.202684 (+ 1994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:18.202692 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:18.202723 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:18.203154 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:18.203168 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.205169 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.205175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:18.207358 (+ 2183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:18.207374 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.208910 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.208916 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:18.209113 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:18.209207 (+ 94us) client_negotiation.cc:769] Sending connection context
0925 09:31:18.210314 (+ 1107us) client_negotiation.cc:240] Negotiation successful
0925 09:31:18.210467 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8391,"mutex_wait_us":74,"thread_start_us":1460,"threads_started":1}
I20260925 09:31:18.210939 28330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:18.200458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55354 (local address 127.10.114.65:39587)
0925 09:31:18.202568 (+ 2110us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:18.202577 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:18.202943 (+ 366us) server_negotiation.cc:407] Connection header received
0925 09:31:18.203100 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:18.203111 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:18.203426 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:18.203625 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:18.204229 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.205732 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.208063 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.209551 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.209688 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:18.209954 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:18.210436 (+ 482us) server_negotiation.cc:299] Negotiation successful
0925 09:31:18.210599 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1923,"thread_start_us":1752,"threads_started":1}
I20260925 09:31:18.211385 28325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:18.199978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:39587 (local address 127.0.0.1:55354)
0925 09:31:18.202800 (+ 2822us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:18.202823 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:18.202947 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:18.203607 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:18.203612 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:18.203636 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:18.204026 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:18.204035 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.205862 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.205867 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:18.207920 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:18.207931 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.210082 (+ 2151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.210088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:18.210235 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:18.210265 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 09:31:18.210335 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 09:31:18.211061 (+ 726us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2700,"mutex_wait_us":619,"thread_start_us":604,"threads_started":1}
I20260925 09:31:18.211995 28326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:18.191538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:44533 (local address 127.0.0.1:45110)
0925 09:31:18.192652 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:18.192692 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:18.192836 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:18.204124 (+ 11288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:18.204130 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:18.204155 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:18.204482 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:18.204493 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.206171 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.206176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:18.208023 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:18.208037 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.209953 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.209959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:18.210148 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:18.210183 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 09:31:18.210246 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 09:31:18.211652 (+ 1406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":868,"mutex_wait_us":1220,"thread_start_us":11007,"threads_started":1}
I20260925 09:31:18.212623 28327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:18.192652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45110 (local address 127.10.114.67:44533)
0925 09:31:18.203562 (+ 10910us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:18.203571 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:18.203602 (+ 31us) server_negotiation.cc:407] Connection header received
0925 09:31:18.203686 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:18.203693 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:18.203956 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:18.204122 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:18.204665 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.206044 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.208179 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.209731 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.210500 (+ 769us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:18.210713 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:18.210776 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 09:31:18.212193 (+ 1417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10743,"thread_start_us":10533,"threads_started":1}
I20260925 09:31:18.214859 28115 tablet_service.cc:1467] Processing CreateTablet for tablet b9309ef4ccfe41d3855e7c2a7a5b4c62 (DEFAULT_TABLE table=test-table [id=6f2bbc8c6c364f6f86df59cd1a899a80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:31:18.214859 27981 tablet_service.cc:1467] Processing CreateTablet for tablet b9309ef4ccfe41d3855e7c2a7a5b4c62 (DEFAULT_TABLE table=test-table [id=6f2bbc8c6c364f6f86df59cd1a899a80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:31:18.215322 28248 tablet_service.cc:1467] Processing CreateTablet for tablet b9309ef4ccfe41d3855e7c2a7a5b4c62 (DEFAULT_TABLE table=test-table [id=6f2bbc8c6c364f6f86df59cd1a899a80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:31:18.216315 27981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9309ef4ccfe41d3855e7c2a7a5b4c62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:18.216323 28115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9309ef4ccfe41d3855e7c2a7a5b4c62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:18.216794 28248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9309ef4ccfe41d3855e7c2a7a5b4c62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:18.227612 28331 tablet_bootstrap.cc:492] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1: Bootstrap starting.
I20260925 09:31:18.230575 28331 tablet_bootstrap.cc:654] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:18.231741 28331 log.cc:824] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:18.233203 28333 tablet_bootstrap.cc:492] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd: Bootstrap starting.
I20260925 09:31:18.233915 28332 tablet_bootstrap.cc:492] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b: Bootstrap starting.
I20260925 09:31:18.234894 28331 tablet_bootstrap.cc:492] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1: No bootstrap required, opened a new log
I20260925 09:31:18.235167 28331 ts_tablet_manager.cc:1397] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260925 09:31:18.237337 28332 tablet_bootstrap.cc:654] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:18.238251 28333 tablet_bootstrap.cc:654] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:18.238507 28332 log.cc:824] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:18.239313 28333 log.cc:824] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:18.241385 28332 tablet_bootstrap.cc:492] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b: No bootstrap required, opened a new log
I20260925 09:31:18.241636 28332 ts_tablet_manager.cc:1397] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260925 09:31:18.241823 28333 tablet_bootstrap.cc:492] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd: No bootstrap required, opened a new log
I20260925 09:31:18.242115 28333 ts_tablet_manager.cc:1397] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260925 09:31:18.245323 28331 raft_consensus.cc:348] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40c4e615d99a48b592818053114667b1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44533 } } peers { permanent_uuid: "157169846f494903bab802b8a6f8f7cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39853 } } peers { permanent_uuid: "c1c0b1afb5244e2684c04c28e1ab2b9b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39587 } }
I20260925 09:31:18.245726 28331 raft_consensus.cc:374] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:18.245820 28331 raft_consensus.cc:729] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40c4e615d99a48b592818053114667b1, State: Initialized, Role: FOLLOWER
I20260925 09:31:18.246400 28331 consensus_queue.cc:260] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40c4e615d99a48b592818053114667b1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44533 } } peers { permanent_uuid: "157169846f494903bab802b8a6f8f7cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39853 } } peers { permanent_uuid: "c1c0b1afb5244e2684c04c28e1ab2b9b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39587 } }
I20260925 09:31:18.247527 28313 heartbeater.cc:503] Master 127.10.114.126:40829 was elected leader, sending a full tablet report...
I20260925 09:31:18.248194 28331 ts_tablet_manager.cc:1428] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1: Time spent starting tablet: real 0.013s user 0.011s sys 0.004s
I20260925 09:31:18.250061 28333 raft_consensus.cc:348] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40c4e615d99a48b592818053114667b1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44533 } } peers { permanent_uuid: "157169846f494903bab802b8a6f8f7cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39853 } } peers { permanent_uuid: "c1c0b1afb5244e2684c04c28e1ab2b9b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39587 } }
I20260925 09:31:18.250388 28333 raft_consensus.cc:374] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:18.250515 28333 raft_consensus.cc:729] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 157169846f494903bab802b8a6f8f7cd, State: Initialized, Role: FOLLOWER
I20260925 09:31:18.251083 28333 consensus_queue.cc:260] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40c4e615d99a48b592818053114667b1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44533 } } peers { permanent_uuid: "157169846f494903bab802b8a6f8f7cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39853 } } peers { permanent_uuid: "c1c0b1afb5244e2684c04c28e1ab2b9b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39587 } }
I20260925 09:31:18.251623 28332 raft_consensus.cc:348] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40c4e615d99a48b592818053114667b1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44533 } } peers { permanent_uuid: "157169846f494903bab802b8a6f8f7cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39853 } } peers { permanent_uuid: "c1c0b1afb5244e2684c04c28e1ab2b9b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39587 } }
I20260925 09:31:18.252018 28332 raft_consensus.cc:374] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:18.252125 28332 raft_consensus.cc:729] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1c0b1afb5244e2684c04c28e1ab2b9b, State: Initialized, Role: FOLLOWER
I20260925 09:31:18.252676 28332 consensus_queue.cc:260] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40c4e615d99a48b592818053114667b1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44533 } } peers { permanent_uuid: "157169846f494903bab802b8a6f8f7cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39853 } } peers { permanent_uuid: "c1c0b1afb5244e2684c04c28e1ab2b9b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39587 } }
I20260925 09:31:18.254336 28332 ts_tablet_manager.cc:1428] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260925 09:31:18.255187 28180 heartbeater.cc:503] Master 127.10.114.126:40829 was elected leader, sending a full tablet report...
I20260925 09:31:18.255748 28333 ts_tablet_manager.cc:1428] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260925 09:31:18.376691 28181 tablet.cc:2369] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:31:18.376950 28314 tablet.cc:2369] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:31:18.398152 28047 tablet.cc:2369] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:31:18.408958 28338 raft_consensus.cc:482] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:31:18.409238 28338 raft_consensus.cc:504] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40c4e615d99a48b592818053114667b1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44533 } } peers { permanent_uuid: "157169846f494903bab802b8a6f8f7cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39853 } } peers { permanent_uuid: "c1c0b1afb5244e2684c04c28e1ab2b9b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39587 } }
I20260925 09:31:18.410871 28338 leader_election.cc:288] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40c4e615d99a48b592818053114667b1 (127.10.114.67:44533), c1c0b1afb5244e2684c04c28e1ab2b9b (127.10.114.65:39587)
I20260925 09:31:18.420223 28330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:18.411411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:40061 (local address 127.10.114.65:39587)
0925 09:31:18.411622 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:18.411630 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:18.412254 (+ 624us) server_negotiation.cc:407] Connection header received
0925 09:31:18.412503 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:18.412507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:18.412607 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:18.412696 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:18.413402 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.414544 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.417801 (+ 3257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.419025 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.419183 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:18.419259 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:18.419844 (+ 585us) server_negotiation.cc:299] Negotiation successful
0925 09:31:18.419969 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 09:31:18.420243 28327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:18.411311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:48503 (local address 127.10.114.67:44533)
0925 09:31:18.411555 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:18.411563 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:18.412560 (+ 997us) server_negotiation.cc:407] Connection header received
0925 09:31:18.412675 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:18.412682 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:18.412752 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:18.412837 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:18.413506 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.414574 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.417794 (+ 3220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.419025 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.419183 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:18.419281 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:18.419849 (+ 568us) server_negotiation.cc:299] Negotiation successful
0925 09:31:18.419969 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 09:31:18.420485 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:18.411310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:39587 (local address 127.10.114.66:40061)
0925 09:31:18.412137 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:18.412152 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:18.412349 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:18.412714 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:18.412717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:18.412733 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:18.413241 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:18.413258 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.414700 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.414705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:18.417612 (+ 2907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:18.417627 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.419272 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.419277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:18.419602 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:18.419696 (+ 94us) client_negotiation.cc:769] Sending connection context
0925 09:31:18.419818 (+ 122us) client_negotiation.cc:240] Negotiation successful
0925 09:31:18.420021 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":472,"thread_start_us":496,"threads_started":1}
I20260925 09:31:18.420529 28341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:18.411644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:44533 (local address 127.10.114.66:48503)
0925 09:31:18.412458 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:18.412476 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:18.412579 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:18.412877 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:18.412881 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:18.412902 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:18.413363 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:18.413373 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.414697 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.414703 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:18.417612 (+ 2909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:18.417627 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.419268 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.419274 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:18.419596 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:18.419695 (+ 99us) client_negotiation.cc:769] Sending connection context
0925 09:31:18.419817 (+ 122us) client_negotiation.cc:240] Negotiation successful
0925 09:31:18.420021 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"mutex_wait_us":354,"thread_start_us":471,"threads_started":1}
I20260925 09:31:18.421535 28268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9309ef4ccfe41d3855e7c2a7a5b4c62" candidate_uuid: "157169846f494903bab802b8a6f8f7cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40c4e615d99a48b592818053114667b1" is_pre_election: true
I20260925 09:31:18.421535 28001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9309ef4ccfe41d3855e7c2a7a5b4c62" candidate_uuid: "157169846f494903bab802b8a6f8f7cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1c0b1afb5244e2684c04c28e1ab2b9b" is_pre_election: true
I20260925 09:31:18.422101 28268 raft_consensus.cc:2454] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 157169846f494903bab802b8a6f8f7cd in term 0.
I20260925 09:31:18.422097 28001 raft_consensus.cc:2454] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 157169846f494903bab802b8a6f8f7cd in term 0.
I20260925 09:31:18.422832 28068 leader_election.cc:302] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 157169846f494903bab802b8a6f8f7cd, c1c0b1afb5244e2684c04c28e1ab2b9b; no voters:
I20260925 09:31:18.423285 28338 raft_consensus.cc:2781] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:31:18.423414 28338 raft_consensus.cc:482] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:31:18.423552 28338 raft_consensus.cc:3037] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:18.425798 28338 raft_consensus.cc:504] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40c4e615d99a48b592818053114667b1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44533 } } peers { permanent_uuid: "157169846f494903bab802b8a6f8f7cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39853 } } peers { permanent_uuid: "c1c0b1afb5244e2684c04c28e1ab2b9b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39587 } }
I20260925 09:31:18.426362 28338 leader_election.cc:288] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [CANDIDATE]: Term 1 election: Requested vote from peers 40c4e615d99a48b592818053114667b1 (127.10.114.67:44533), c1c0b1afb5244e2684c04c28e1ab2b9b (127.10.114.65:39587)
I20260925 09:31:18.426851 28268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9309ef4ccfe41d3855e7c2a7a5b4c62" candidate_uuid: "157169846f494903bab802b8a6f8f7cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40c4e615d99a48b592818053114667b1"
I20260925 09:31:18.426851 28001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9309ef4ccfe41d3855e7c2a7a5b4c62" candidate_uuid: "157169846f494903bab802b8a6f8f7cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1c0b1afb5244e2684c04c28e1ab2b9b"
I20260925 09:31:18.427057 28001 raft_consensus.cc:3037] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:18.427057 28268 raft_consensus.cc:3037] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:18.430269 28001 raft_consensus.cc:2454] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 157169846f494903bab802b8a6f8f7cd in term 1.
I20260925 09:31:18.430266 28268 raft_consensus.cc:2454] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 157169846f494903bab802b8a6f8f7cd in term 1.
I20260925 09:31:18.430657 28068 leader_election.cc:302] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 157169846f494903bab802b8a6f8f7cd, c1c0b1afb5244e2684c04c28e1ab2b9b; no voters:
I20260925 09:31:18.430977 28338 raft_consensus.cc:2781] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:18.431375 28338 raft_consensus.cc:686] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [term 1 LEADER]: Becoming Leader. State: Replica: 157169846f494903bab802b8a6f8f7cd, State: Running, Role: LEADER
I20260925 09:31:18.431880 28338 consensus_queue.cc:237] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40c4e615d99a48b592818053114667b1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44533 } } peers { permanent_uuid: "157169846f494903bab802b8a6f8f7cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39853 } } peers { permanent_uuid: "c1c0b1afb5244e2684c04c28e1ab2b9b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39587 } }
I20260925 09:31:18.436590 27856 catalog_manager.cc:5495] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 157169846f494903bab802b8a6f8f7cd (127.10.114.66). New cstate: current_term: 1 leader_uuid: "157169846f494903bab802b8a6f8f7cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40c4e615d99a48b592818053114667b1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "157169846f494903bab802b8a6f8f7cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 39853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1c0b1afb5244e2684c04c28e1ab2b9b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39587 } health_report { overall_health: UNKNOWN } } }
I20260925 09:31:18.497766 28328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:18.492479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47900 (local address 127.10.114.66:39853)
0925 09:31:18.492670 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:18.492674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:18.492759 (+ 85us) server_negotiation.cc:407] Connection header received
0925 09:31:18.492928 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:18.492932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:18.493006 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:18.493127 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:31:18.493519 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.494396 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.495852 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.496248 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.496343 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:18.496777 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:31:18.497283 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:31:18.497399 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:18.497490 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 09:31:18.497555 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:31:18.508440 28001 raft_consensus.cc:1261] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b [term 1 FOLLOWER]: Refusing update from remote peer 157169846f494903bab802b8a6f8f7cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:31:18.508586 28268 raft_consensus.cc:1261] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1 [term 1 FOLLOWER]: Refusing update from remote peer 157169846f494903bab802b8a6f8f7cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:31:18.509464 28342 consensus_queue.cc:1035] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40c4e615d99a48b592818053114667b1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:31:18.509812 28338 consensus_queue.cc:1035] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1c0b1afb5244e2684c04c28e1ab2b9b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 39587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:31:18.545507 27856 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:31:18.545807 27856 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:31:18.549750 27856 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 947f42665232458dbc8ea44232a1248e: Sending DeleteTablet for 3 replicas of tablet b9309ef4ccfe41d3855e7c2a7a5b4c62
I20260925 09:31:18.550781 28248 tablet_service.cc:1514] Processing DeleteTablet for tablet b9309ef4ccfe41d3855e7c2a7a5b4c62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:31:18 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:45110
I20260925 09:31:18.550869 28115 tablet_service.cc:1514] Processing DeleteTablet for tablet b9309ef4ccfe41d3855e7c2a7a5b4c62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:31:18 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:47886
I20260925 09:31:18.551088 27981 tablet_service.cc:1514] Processing DeleteTablet for tablet b9309ef4ccfe41d3855e7c2a7a5b4c62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:31:18 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:55354
I20260925 09:31:18.553601 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 27918
I20260925 09:31:18.564540 28357 tablet_replica.cc:330] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b: stopping tablet replica
I20260925 09:31:18.565035 28355 tablet_replica.cc:330] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1: stopping tablet replica
I20260925 09:31:18.565343 28356 tablet_replica.cc:330] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd: stopping tablet replica
I20260925 09:31:18.565747 28356 raft_consensus.cc:2229] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:31:18.566100 28355 raft_consensus.cc:2229] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:31:18.566454 28356 raft_consensus.cc:2258] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:31:18.565797 28357 raft_consensus.cc:2229] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:31:18.567984 28357 raft_consensus.cc:2258] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:31:18.568238 28355 raft_consensus.cc:2258] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:31:18.568596 28356 ts_tablet_manager.cc:1905] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:31:18.570573 28355 ts_tablet_manager.cc:1905] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:31:18.571470 28357 ts_tablet_manager.cc:1905] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 25 09:31:18 dist-test-slave-9pj8 krb5kdc[27812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:31:18.578786 28357 ts_tablet_manager.cc:1918] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:31:18.579026 28357 log.cc:1196] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/b9309ef4ccfe41d3855e7c2a7a5b4c62
I20260925 09:31:18.579526 28355 ts_tablet_manager.cc:1918] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:31:18.579596 28357 ts_tablet_manager.cc:1939] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P c1c0b1afb5244e2684c04c28e1ab2b9b: Deleting consensus metadata
I20260925 09:31:18.580250 28356 ts_tablet_manager.cc:1918] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:31:18.580430 28356 log.cc:1196] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/b9309ef4ccfe41d3855e7c2a7a5b4c62
I20260925 09:31:18.580968 28356 ts_tablet_manager.cc:1939] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 157169846f494903bab802b8a6f8f7cd: Deleting consensus metadata
I20260925 09:31:18.581008 28355 log.cc:1196] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/b9309ef4ccfe41d3855e7c2a7a5b4c62
I20260925 09:31:18.582170 27841 catalog_manager.cc:4841] TS c1c0b1afb5244e2684c04c28e1ab2b9b (127.10.114.65:39587): tablet b9309ef4ccfe41d3855e7c2a7a5b4c62 (table test-table [id=6f2bbc8c6c364f6f86df59cd1a899a80]) successfully deleted
I20260925 09:31:18.582695 27844 catalog_manager.cc:4841] TS 157169846f494903bab802b8a6f8f7cd (127.10.114.66:39853): tablet b9309ef4ccfe41d3855e7c2a7a5b4c62 (table test-table [id=6f2bbc8c6c364f6f86df59cd1a899a80]) successfully deleted
I20260925 09:31:18.583134 28355 ts_tablet_manager.cc:1939] T b9309ef4ccfe41d3855e7c2a7a5b4c62 P 40c4e615d99a48b592818053114667b1: Deleting consensus metadata
I20260925 09:31:18.584517 27843 catalog_manager.cc:4841] TS 40c4e615d99a48b592818053114667b1 (127.10.114.67:44533): tablet b9309ef4ccfe41d3855e7c2a7a5b4c62 (table test-table [id=6f2bbc8c6c364f6f86df59cd1a899a80]) successfully deleted
I20260925 09:31:18.586840 28330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:18.555258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55360 (local address 127.10.114.65:39587)
0925 09:31:18.555496 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:18.555505 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:18.560145 (+ 4640us) server_negotiation.cc:407] Connection header received
0925 09:31:18.560337 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:18.560346 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:18.560449 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:18.560572 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:18.566133 (+ 5561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.567360 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.569266 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.569856 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.569974 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:18.579992 (+ 10018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:18.580006 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:18.580024 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:18.580064 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:18.583878 (+ 3814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:18.584789 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:18.584796 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:18.584798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:18.584917 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:18.585373 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:18.585376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:18.585378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:18.585685 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:18.585898 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:18.586313 (+ 415us) server_negotiation.cc:299] Negotiation successful
0925 09:31:18.586479 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 09:31:18.587561 28041 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:18.694419 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 27918
I20260925 09:31:18.713573 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 28052
Sep 25 09:31:18 dist-test-slave-9pj8 krb5kdc[27812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.66@KRBTEST.COM
I20260925 09:31:18.731082 28328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:18.714356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47902 (local address 127.10.114.66:39853)
0925 09:31:18.714543 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:18.714547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:18.714591 (+ 44us) server_negotiation.cc:407] Connection header received
0925 09:31:18.714811 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:18.714815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:18.714880 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:18.714962 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:18.716645 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.717479 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.719369 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.719915 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.720026 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:18.725124 (+ 5098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:18.725138 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:18.725154 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:18.725190 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:18.728274 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:18.728985 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:18.728997 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:18.728999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:18.729133 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:18.729514 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:18.729520 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:18.729533 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:18.729887 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:18.730243 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:18.730488 (+ 245us) server_negotiation.cc:299] Negotiation successful
0925 09:31:18.730720 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 09:31:18.731887 28175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:18.843886 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 28052
I20260925 09:31:18.862577 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 28185
Sep 25 09:31:18 dist-test-slave-9pj8 krb5kdc[27812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.67@KRBTEST.COM
I20260925 09:31:18.877494 28327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:18.863434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45116 (local address 127.10.114.67:44533)
0925 09:31:18.863617 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:18.863621 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:18.863641 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:31:18.863742 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:18.863746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:18.863889 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:18.863988 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:18.865147 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.866026 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.867204 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:18.867567 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:18.867659 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:18.871888 (+ 4229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:18.871903 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:18.871924 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:18.871955 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:18.874561 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:18.875347 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:18.875476 (+ 129us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:18.875482 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:18.875654 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:18.876121 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:18.876125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:18.876127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:18.876481 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:18.876718 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:18.876966 (+ 248us) server_negotiation.cc:299] Negotiation successful
0925 09:31:18.877129 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 09:31:18.878144 28308 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:18.965248 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 28185
I20260925 09:31:18.982095 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 27825
I20260925 09:31:18.983235 27887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:19.091905 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 27825
2026-09-25T09:31:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4299 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-25T09:31:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:31:19Z Disabled control of system clock
I20260925 09:31:19.166734 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:40645
--webserver_interface=127.10.114.126
--webserver_port=0
--builtin_ntp_servers=127.10.114.84:40753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:40645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 09:31:19.484730 28366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:19.484995 28366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:19.485118 28366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:19.492282 28366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:31:19.492399 28366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:19.492480 28366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:19.492594 28366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:31:19.492646 28366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:31:19.501847 28366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:40753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:40645
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:40645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:19.503039 28366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:19.504830 28366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:19.516788 28371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:19.516700 28372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:19.516752 28374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:19.517910 28366 server_base.cc:1034] running on GCE node
I20260925 09:31:19.518669 28366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:19.520108 28366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:19.521335 28366 hybrid_clock.cc:648] HybridClock initialized: now 1790328679521281 us; error 55 us; skew 500 ppm
I20260925 09:31:19.521665 28366 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:31:19.523921 28366 webserver.cc:460] Webserver started at http://127.10.114.126:41539/ using document root <none> and password file <none>
I20260925 09:31:19.524590 28366 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:19.524708 28366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:19.525027 28366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:19.527701 28366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "6f2123cdea3e448a8b22688d2ab25020"
format_stamp: "Formatted at 2026-09-25 09:31:19 on dist-test-slave-9pj8"
server_key: "c3e0460dde678d2c56f010320939b336"
server_key_iv: "b2f0a2f17c8b1f32042bfe18a93bd39e"
server_key_version: "encryptionkey@0"
I20260925 09:31:19.528555 28366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "6f2123cdea3e448a8b22688d2ab25020"
format_stamp: "Formatted at 2026-09-25 09:31:19 on dist-test-slave-9pj8"
server_key: "c3e0460dde678d2c56f010320939b336"
server_key_iv: "b2f0a2f17c8b1f32042bfe18a93bd39e"
server_key_version: "encryptionkey@0"
I20260925 09:31:19.533249 28366 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:31:19.536468 28380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:19.537734 28366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:31:19.537916 28366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "6f2123cdea3e448a8b22688d2ab25020"
format_stamp: "Formatted at 2026-09-25 09:31:19 on dist-test-slave-9pj8"
server_key: "c3e0460dde678d2c56f010320939b336"
server_key_iv: "b2f0a2f17c8b1f32042bfe18a93bd39e"
server_key_version: "encryptionkey@0"
I20260925 09:31:19.538120 28366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:19.549667 28366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:19.550372 28366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:19.550633 28366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:19.568379 28366 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:40645
I20260925 09:31:19.568397 28431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:40645 every 8 connection(s)
I20260925 09:31:19.569526 28366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:31:19.572842 28432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:19.576035 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 28366
I20260925 09:31:19.576232 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:31:19.576687 10697 external_mini_cluster.cc:1428] Setting key e9ca6c27f44da7067cda3a182313991c
I20260925 09:31:19.584925 28432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020: Bootstrap starting.
I20260925 09:31:19.588493 28432 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:19.589629 28432 log.cc:824] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:19.590334 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:19.578519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47890 (local address 127.10.114.126:40645)
0925 09:31:19.579497 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:19.579507 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:19.579547 (+ 40us) server_negotiation.cc:407] Connection header received
0925 09:31:19.581174 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:19.581199 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:19.581660 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:19.582009 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:19.582606 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:19.583978 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:19.585848 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:19.586683 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:19.586822 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:19.587989 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:19.587996 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:19.588008 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:19.588477 (+ 469us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:19.588597 (+ 120us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:19.588656 (+ 59us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:19.588750 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:19.588970 (+ 220us) server_negotiation.cc:299] Negotiation successful
0925 09:31:19.589259 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":377,"threads_started":1}
I20260925 09:31:19.592453 28432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020: No bootstrap required, opened a new log
I20260925 09:31:19.599195 28432 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2123cdea3e448a8b22688d2ab25020" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40645 } }
I20260925 09:31:19.599536 28432 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:19.599635 28432 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f2123cdea3e448a8b22688d2ab25020, State: Initialized, Role: FOLLOWER
I20260925 09:31:19.600163 28432 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2123cdea3e448a8b22688d2ab25020" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40645 } }
I20260925 09:31:19.600407 28432 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:31:19.600544 28432 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:31:19.600720 28432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:19.602698 28432 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2123cdea3e448a8b22688d2ab25020" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40645 } }
I20260925 09:31:19.603163 28432 leader_election.cc:302] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f2123cdea3e448a8b22688d2ab25020; no voters:
I20260925 09:31:19.603832 28432 leader_election.cc:288] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:31:19.604001 28437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:19.604954 28437 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020 [term 1 LEADER]: Becoming Leader. State: Replica: 6f2123cdea3e448a8b22688d2ab25020, State: Running, Role: LEADER
I20260925 09:31:19.605464 28437 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2123cdea3e448a8b22688d2ab25020" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40645 } }
I20260925 09:31:19.605829 28432 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:31:19.608349 28439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f2123cdea3e448a8b22688d2ab25020. Latest consensus state: current_term: 1 leader_uuid: "6f2123cdea3e448a8b22688d2ab25020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2123cdea3e448a8b22688d2ab25020" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40645 } } }
I20260925 09:31:19.608296 28438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f2123cdea3e448a8b22688d2ab25020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2123cdea3e448a8b22688d2ab25020" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 40645 } } }
I20260925 09:31:19.608793 28438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:19.608790 28439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:19.611858 28445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:31:19.616763 28445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:31:19.625733 28445 catalog_manager.cc:1347] Generated new cluster ID: 4836b5272e4c436f8870222708e0d7d7
I20260925 09:31:19.625900 28445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:31:19.640709 28445 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:31:19.642190 28445 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:31:19.649837 28445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020: Generated new TSK 0
I20260925 09:31:19.650887 28445 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:31:19.668177 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:40645
--builtin_ntp_servers=127.10.114.84:40753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 09:31:19.988507 28456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:19.988811 28456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:19.988893 28456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:19.995509 28456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:19.995586 28456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:19.995678 28456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:31:20.004086 28456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:40753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:40645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:20.005121 28456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:20.006954 28456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:20.020901 28462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:20.020977 28461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:20.020952 28464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:20.021842 28456 server_base.cc:1034] running on GCE node
I20260925 09:31:20.022502 28456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:20.023478 28456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:20.024667 28456 hybrid_clock.cc:648] HybridClock initialized: now 1790328680024632 us; error 27 us; skew 500 ppm
I20260925 09:31:20.024981 28456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:31:20.026997 28456 webserver.cc:460] Webserver started at http://127.10.114.65:45895/ using document root <none> and password file <none>
I20260925 09:31:20.027635 28456 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:20.027748 28456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:20.028098 28456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:20.030753 28456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "445a59abeb8f4c8db0a3b4b8a330b6e4"
format_stamp: "Formatted at 2026-09-25 09:31:20 on dist-test-slave-9pj8"
server_key: "2e6fd026f66f50530ef5015d2c950a6c"
server_key_iv: "640aa59bf61842879de279592cd8d82b"
server_key_version: "encryptionkey@0"
I20260925 09:31:20.031493 28456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "445a59abeb8f4c8db0a3b4b8a330b6e4"
format_stamp: "Formatted at 2026-09-25 09:31:20 on dist-test-slave-9pj8"
server_key: "2e6fd026f66f50530ef5015d2c950a6c"
server_key_iv: "640aa59bf61842879de279592cd8d82b"
server_key_version: "encryptionkey@0"
I20260925 09:31:20.036439 28456 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260925 09:31:20.039835 28470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:20.041049 28456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 09:31:20.041229 28456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "445a59abeb8f4c8db0a3b4b8a330b6e4"
format_stamp: "Formatted at 2026-09-25 09:31:20 on dist-test-slave-9pj8"
server_key: "2e6fd026f66f50530ef5015d2c950a6c"
server_key_iv: "640aa59bf61842879de279592cd8d82b"
server_key_version: "encryptionkey@0"
I20260925 09:31:20.041424 28456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:20.056986 28456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:20.057621 28456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:20.057905 28456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:20.058763 28456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:31:20.060246 28456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:31:20.060364 28456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:20.060482 28456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:31:20.060559 28456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:20.092890 28456 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:34015
I20260925 09:31:20.092975 28582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:34015 every 8 connection(s)
I20260925 09:31:20.094125 28456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:31:20.100473 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 28456
I20260925 09:31:20.100647 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:31:20.101075 10697 external_mini_cluster.cc:1428] Setting key 0445fa0cdc457a7924df2b7706bf2046
I20260925 09:31:20.103478 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:40645
--builtin_ntp_servers=127.10.114.84:40753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 09:31:20.106094 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:20.096689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:40791 (local address 127.10.114.126:40645)
0925 09:31:20.097161 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:20.097166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:20.098069 (+ 903us) server_negotiation.cc:407] Connection header received
0925 09:31:20.099743 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:20.099748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:20.099853 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:20.099968 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:20.101095 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:20.101838 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:20.103295 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:20.103829 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:20.103952 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:20.105295 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:20.105300 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:20.105303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:20.105319 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:20.105358 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:20.105369 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:20.105459 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:20.105662 (+ 203us) server_negotiation.cc:299] Negotiation successful
0925 09:31:20.105762 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":260,"threads_started":1}
I20260925 09:31:20.107034 28586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:20.097122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40645 (local address 127.10.114.65:40791)
0925 09:31:20.097923 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:20.097956 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:20.099475 (+ 1519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:20.100190 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:20.100197 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:20.100280 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:31:20.100940 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:20.100957 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:20.101990 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:20.101994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:20.103148 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:20.103160 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:20.104049 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:20.104057 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:20.104282 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:20.104989 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:31:20.104991 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:31:20.105028 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:31:20.105100 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:31:20.105458 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:31:20.105471 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:31:20.105481 (+ 10us) client_negotiation.cc:769] Sending connection context
0925 09:31:20.105652 (+ 171us) client_negotiation.cc:240] Negotiation successful
0925 09:31:20.106018 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":232,"threads_started":1}
I20260925 09:31:20.108574 28583 heartbeater.cc:347] Connected to a master server at 127.10.114.126:40645
I20260925 09:31:20.108844 28583 heartbeater.cc:464] Registering TS with master...
I20260925 09:31:20.109505 28583 heartbeater.cc:511] Master 127.10.114.126:40645 requested a full tablet report, sending...
I20260925 09:31:20.111743 28397 ts_manager.cc:194] Registered new tserver with Master: 445a59abeb8f4c8db0a3b4b8a330b6e4 (127.10.114.65:34015)
I20260925 09:31:20.113222 28397 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.114.65:40791
W20260925 09:31:20.416466 28587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:20.416749 28587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:20.416904 28587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:20.423914 28587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:20.424042 28587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:20.424203 28587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:31:20.432956 28587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:40753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:40645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:20.434172 28587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:20.435909 28587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:20.448539 28592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:20.448589 28593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:20.448553 28595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:20.449729 28587 server_base.cc:1034] running on GCE node
I20260925 09:31:20.450237 28587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:20.451043 28587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:20.452229 28587 hybrid_clock.cc:648] HybridClock initialized: now 1790328680452196 us; error 56 us; skew 500 ppm
I20260925 09:31:20.452538 28587 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:31:20.454648 28587 webserver.cc:460] Webserver started at http://127.10.114.66:43777/ using document root <none> and password file <none>
I20260925 09:31:20.455282 28587 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:20.455390 28587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:20.455704 28587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:20.458439 28587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "e3663102c98748d88b2218aa1768e96c"
format_stamp: "Formatted at 2026-09-25 09:31:20 on dist-test-slave-9pj8"
server_key: "4c853d942e2c4ef91f18fcc2f8b9673c"
server_key_iv: "614a14356c4b82db0fa25b3710ef1e29"
server_key_version: "encryptionkey@0"
I20260925 09:31:20.459235 28587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "e3663102c98748d88b2218aa1768e96c"
format_stamp: "Formatted at 2026-09-25 09:31:20 on dist-test-slave-9pj8"
server_key: "4c853d942e2c4ef91f18fcc2f8b9673c"
server_key_iv: "614a14356c4b82db0fa25b3710ef1e29"
server_key_version: "encryptionkey@0"
I20260925 09:31:20.464133 28587 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 09:31:20.467403 28601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:20.468719 28587 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:31:20.468874 28587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "e3663102c98748d88b2218aa1768e96c"
format_stamp: "Formatted at 2026-09-25 09:31:20 on dist-test-slave-9pj8"
server_key: "4c853d942e2c4ef91f18fcc2f8b9673c"
server_key_iv: "614a14356c4b82db0fa25b3710ef1e29"
server_key_version: "encryptionkey@0"
I20260925 09:31:20.469071 28587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:20.496016 28587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:20.496767 28587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:20.497056 28587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:20.498030 28587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:31:20.500155 28587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:31:20.500275 28587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:20.500396 28587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:31:20.500490 28587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:20.536646 28587 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:41355
I20260925 09:31:20.536739 28713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:41355 every 8 connection(s)
I20260925 09:31:20.537909 28587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:31:20.546444 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 28587
I20260925 09:31:20.546640 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:31:20.547055 10697 external_mini_cluster.cc:1428] Setting key 66af17be040664d33532d6e8d2934d16
I20260925 09:31:20.549938 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:40645
--builtin_ntp_servers=127.10.114.84:40753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 09:31:20.550706 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:20.540564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:48381 (local address 127.10.114.126:40645)
0925 09:31:20.540743 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:20.540748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:20.542137 (+ 1389us) server_negotiation.cc:407] Connection header received
0925 09:31:20.544048 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:20.544056 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:20.544131 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:20.544218 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:20.545393 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:20.546094 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:20.547825 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:20.548410 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:20.548556 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:20.549848 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:20.549853 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:20.549856 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:20.549872 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:20.549901 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:20.549913 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:20.550008 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:20.550278 (+ 270us) server_negotiation.cc:299] Negotiation successful
0925 09:31:20.550371 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 09:31:20.551584 28716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:20.541058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40645 (local address 127.10.114.66:48381)
0925 09:31:20.542007 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:20.542049 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:20.543692 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:20.544485 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:20.544493 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:20.544578 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:31:20.545205 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:20.545227 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:20.546244 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:20.546250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:20.547650 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:20.547662 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:20.548636 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:20.548643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:20.548866 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:20.549545 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:31:20.549547 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:31:20.549589 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:31:20.549658 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:31:20.550008 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:31:20.550021 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:31:20.550030 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 09:31:20.550245 (+ 215us) client_negotiation.cc:240] Negotiation successful
0925 09:31:20.550604 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":265,"threads_started":1}
I20260925 09:31:20.552985 28714 heartbeater.cc:347] Connected to a master server at 127.10.114.126:40645
I20260925 09:31:20.553263 28714 heartbeater.cc:464] Registering TS with master...
I20260925 09:31:20.553941 28714 heartbeater.cc:511] Master 127.10.114.126:40645 requested a full tablet report, sending...
I20260925 09:31:20.555452 28397 ts_manager.cc:194] Registered new tserver with Master: e3663102c98748d88b2218aa1768e96c (127.10.114.66:41355)
I20260925 09:31:20.556648 28397 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.114.66:48381
W20260925 09:31:20.874284 28717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:20.874552 28717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:20.874652 28717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:20.881844 28717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:20.881942 28717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:20.882045 28717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:31:20.890727 28717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:40753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:40645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:20.891978 28717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:20.893787 28717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:20.906402 28725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:20.906378 28722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:20.906374 28723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:20.907090 28717 server_base.cc:1034] running on GCE node
I20260925 09:31:20.907913 28717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:20.908809 28717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:20.910007 28717 hybrid_clock.cc:648] HybridClock initialized: now 1790328680909957 us; error 42 us; skew 500 ppm
I20260925 09:31:20.910360 28717 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:31:20.912628 28717 webserver.cc:460] Webserver started at http://127.10.114.67:35503/ using document root <none> and password file <none>
I20260925 09:31:20.913421 28717 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:20.913548 28717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:20.913863 28717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:20.916662 28717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "43d727296b1b416c997624f54525d7e1"
format_stamp: "Formatted at 2026-09-25 09:31:20 on dist-test-slave-9pj8"
server_key: "83d9c294d1e93c813adbda3c16d0af20"
server_key_iv: "cd5311e60f56190d9c25bc66d0c22638"
server_key_version: "encryptionkey@0"
I20260925 09:31:20.917414 28717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "43d727296b1b416c997624f54525d7e1"
format_stamp: "Formatted at 2026-09-25 09:31:20 on dist-test-slave-9pj8"
server_key: "83d9c294d1e93c813adbda3c16d0af20"
server_key_iv: "cd5311e60f56190d9c25bc66d0c22638"
server_key_version: "encryptionkey@0"
I20260925 09:31:20.922679 28717 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:31:20.926106 28731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:20.927439 28717 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260925 09:31:20.927637 28717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "43d727296b1b416c997624f54525d7e1"
format_stamp: "Formatted at 2026-09-25 09:31:20 on dist-test-slave-9pj8"
server_key: "83d9c294d1e93c813adbda3c16d0af20"
server_key_iv: "cd5311e60f56190d9c25bc66d0c22638"
server_key_version: "encryptionkey@0"
I20260925 09:31:20.927891 28717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:20.943027 28717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:20.943956 28717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:20.944258 28717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:20.945319 28717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:31:20.946754 28717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:31:20.946877 28717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:20.947022 28717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:31:20.947117 28717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:20.982137 28717 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:44741
I20260925 09:31:20.982234 28843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:44741 every 8 connection(s)
I20260925 09:31:20.983541 28717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:31:20.984248 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 28717
I20260925 09:31:20.984437 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:31:20.985062 10697 external_mini_cluster.cc:1428] Setting key a9f3e8befbc316ab10f1f0163cfa850a
I20260925 09:31:20.996771 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:20.986384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:54493 (local address 127.10.114.126:40645)
0925 09:31:20.986588 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:20.986594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:20.988124 (+ 1530us) server_negotiation.cc:407] Connection header received
0925 09:31:20.989878 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:20.989883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:20.989952 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:20.990051 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:20.991020 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:20.991841 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:20.994197 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:20.994589 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:20.994703 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:20.996058 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:20.996063 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:20.996066 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:20.996085 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:20.996121 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:20.996134 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:20.996222 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:20.996428 (+ 206us) server_negotiation.cc:299] Negotiation successful
0925 09:31:20.996505 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 09:31:20.997608 28846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:20.986966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:40645 (local address 127.10.114.67:54493)
0925 09:31:20.987984 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:20.988025 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:20.989638 (+ 1613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:20.990253 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:20.990260 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:20.990337 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:31:20.990856 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:20.990871 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:20.992026 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:20.992031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:20.994043 (+ 2012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:20.994056 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:20.994789 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:20.994794 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:20.994975 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:20.995682 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:31:20.995685 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:31:20.995725 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:31:20.995853 (+ 128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:31:20.996249 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:31:20.996257 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:31:20.996262 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 09:31:20.996392 (+ 130us) client_negotiation.cc:240] Negotiation successful
0925 09:31:20.996648 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":339,"threads_started":1}
I20260925 09:31:20.998895 28844 heartbeater.cc:347] Connected to a master server at 127.10.114.126:40645
I20260925 09:31:20.999188 28844 heartbeater.cc:464] Registering TS with master...
I20260925 09:31:20.999948 28844 heartbeater.cc:511] Master 127.10.114.126:40645 requested a full tablet report, sending...
I20260925 09:31:21.001289 28397 ts_manager.cc:194] Registered new tserver with Master: 43d727296b1b416c997624f54525d7e1 (127.10.114.67:44741)
I20260925 09:31:21.002077 28397 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.114.67:54493
I20260925 09:31:21.012609 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:31:21.023401 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:21.016957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47900 (local address 127.10.114.126:40645)
0925 09:31:21.017153 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:21.017161 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:21.017657 (+ 496us) server_negotiation.cc:407] Connection header received
0925 09:31:21.017864 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:21.017870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:21.017964 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:21.018076 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:21.018593 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.019373 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.020975 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.021439 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.021583 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:21.022637 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:21.022643 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:21.022648 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:21.022676 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:21.022717 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:21.022731 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:21.022839 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:21.022991 (+ 152us) server_negotiation.cc:299] Negotiation successful
0925 09:31:21.023089 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 09:31:21.027281 28397 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:31:21.034490 28397 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:31:21.069280 28857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:21.047205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:41355 (local address 127.0.0.1:46532)
0925 09:31:21.059832 (+ 12627us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:21.059872 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:21.060067 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:21.061007 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:21.061014 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:21.061047 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:21.061538 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:21.061553 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.063615 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.063623 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:21.065938 (+ 2315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:21.065955 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.068086 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.068093 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:21.068397 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:21.068461 (+ 64us) client_negotiation.cc:769] Sending connection context
0925 09:31:21.068607 (+ 146us) client_negotiation.cc:240] Negotiation successful
0925 09:31:21.068796 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12371,"mutex_wait_us":26,"thread_start_us":13899,"threads_started":1}
I20260925 09:31:21.069283 28858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:21.047577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46532 (local address 127.10.114.66:41355)
0925 09:31:21.059361 (+ 11784us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:21.059375 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:21.060292 (+ 917us) server_negotiation.cc:407] Connection header received
0925 09:31:21.060426 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:21.060434 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:21.060728 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:21.061577 (+ 849us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:21.061825 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.063472 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.066159 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.067503 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.067652 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:21.067953 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:21.068691 (+ 738us) server_negotiation.cc:299] Negotiation successful
0925 09:31:21.068913 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11616,"thread_start_us":11498,"threads_started":1}
I20260925 09:31:21.070145 28856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:21.047240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:44741 (local address 127.0.0.1:42646)
0925 09:31:21.061744 (+ 14504us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:21.061768 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:21.061898 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:21.062608 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:21.062616 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:21.062651 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:21.063136 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:21.063149 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.065394 (+ 2245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.065400 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:21.067440 (+ 2040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:21.067452 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.068995 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.069003 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:21.069226 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:21.069291 (+ 65us) client_negotiation.cc:769] Sending connection context
0925 09:31:21.069363 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 09:31:21.069498 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14094,"mutex_wait_us":1,"thread_start_us":12187,"threads_started":1}
I20260925 09:31:21.071246 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:21.047669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42646 (local address 127.10.114.67:44741)
0925 09:31:21.061037 (+ 13368us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:21.061046 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:21.061860 (+ 814us) server_negotiation.cc:407] Connection header received
0925 09:31:21.062096 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:21.062106 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:21.062349 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:21.063339 (+ 990us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:21.063595 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.065252 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.067628 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.068745 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.070426 (+ 1681us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:21.070726 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:21.070798 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 09:31:21.070964 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13174,"thread_start_us":12974,"threads_started":1}
I20260925 09:31:21.072410 28861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:21.061596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:34015 (local address 127.0.0.1:41728)
0925 09:31:21.062991 (+ 1395us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:21.063016 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:21.063153 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:21.064066 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:21.064070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:21.064101 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:21.064738 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:21.064752 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.066766 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.066772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:21.069064 (+ 2292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:21.069079 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.071550 (+ 2471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.071555 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:21.071681 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:21.071706 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 09:31:21.071844 (+ 138us) client_negotiation.cc:240] Negotiation successful
0925 09:31:21.071966 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1242,"mutex_wait_us":155,"thread_start_us":315,"threads_started":1}
I20260925 09:31:21.072479 28860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:21.061864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41728 (local address 127.10.114.65:34015)
0925 09:31:21.062508 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:21.062519 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:21.063132 (+ 613us) server_negotiation.cc:407] Connection header received
0925 09:31:21.063327 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:21.063335 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:21.063553 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:21.063753 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:21.065004 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.066608 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.069339 (+ 2731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.070845 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.070972 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:21.071218 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:21.071888 (+ 670us) server_negotiation.cc:299] Negotiation successful
0925 09:31:21.072061 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":312,"threads_started":1}
I20260925 09:31:21.074568 28649 tablet_service.cc:1467] Processing CreateTablet for tablet 244049c7028a4321bc726b99b72f27da (DEFAULT_TABLE table=test-table [id=d9767c4c33134a48aa8cfb1288cdd11c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:31:21.075155 28779 tablet_service.cc:1467] Processing CreateTablet for tablet 244049c7028a4321bc726b99b72f27da (DEFAULT_TABLE table=test-table [id=d9767c4c33134a48aa8cfb1288cdd11c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:31:21.075522 28518 tablet_service.cc:1467] Processing CreateTablet for tablet 244049c7028a4321bc726b99b72f27da (DEFAULT_TABLE table=test-table [id=d9767c4c33134a48aa8cfb1288cdd11c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:31:21.076169 28649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 244049c7028a4321bc726b99b72f27da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:21.076715 28779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 244049c7028a4321bc726b99b72f27da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:21.076804 28518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 244049c7028a4321bc726b99b72f27da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:21.087647 28862 tablet_bootstrap.cc:492] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4: Bootstrap starting.
I20260925 09:31:21.091010 28862 tablet_bootstrap.cc:654] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:21.092324 28862 log.cc:824] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:21.093386 28863 tablet_bootstrap.cc:492] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1: Bootstrap starting.
I20260925 09:31:21.097749 28863 tablet_bootstrap.cc:654] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:21.099469 28863 log.cc:824] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:21.100016 28865 tablet_bootstrap.cc:492] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c: Bootstrap starting.
I20260925 09:31:21.104298 28862 tablet_bootstrap.cc:492] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4: No bootstrap required, opened a new log
I20260925 09:31:21.104664 28862 ts_tablet_manager.cc:1397] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4: Time spent bootstrapping tablet: real 0.017s user 0.013s sys 0.000s
I20260925 09:31:21.105366 28865 tablet_bootstrap.cc:654] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:21.106649 28865 log.cc:824] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:21.110502 28863 tablet_bootstrap.cc:492] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1: No bootstrap required, opened a new log
I20260925 09:31:21.110659 28865 tablet_bootstrap.cc:492] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c: No bootstrap required, opened a new log
I20260925 09:31:21.110913 28865 ts_tablet_manager.cc:1397] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260925 09:31:21.110925 28863 ts_tablet_manager.cc:1397] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.013s
I20260925 09:31:21.115821 28583 heartbeater.cc:503] Master 127.10.114.126:40645 was elected leader, sending a full tablet report...
I20260925 09:31:21.117395 28862 raft_consensus.cc:348] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3663102c98748d88b2218aa1768e96c" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41355 } } peers { permanent_uuid: "43d727296b1b416c997624f54525d7e1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44741 } } peers { permanent_uuid: "445a59abeb8f4c8db0a3b4b8a330b6e4" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34015 } }
I20260925 09:31:21.117882 28862 raft_consensus.cc:374] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:21.118006 28862 raft_consensus.cc:729] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 445a59abeb8f4c8db0a3b4b8a330b6e4, State: Initialized, Role: FOLLOWER
I20260925 09:31:21.118803 28862 consensus_queue.cc:260] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3663102c98748d88b2218aa1768e96c" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41355 } } peers { permanent_uuid: "43d727296b1b416c997624f54525d7e1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44741 } } peers { permanent_uuid: "445a59abeb8f4c8db0a3b4b8a330b6e4" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34015 } }
I20260925 09:31:21.120110 28863 raft_consensus.cc:348] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3663102c98748d88b2218aa1768e96c" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41355 } } peers { permanent_uuid: "43d727296b1b416c997624f54525d7e1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44741 } } peers { permanent_uuid: "445a59abeb8f4c8db0a3b4b8a330b6e4" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34015 } }
I20260925 09:31:21.120627 28863 raft_consensus.cc:374] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:21.120796 28863 raft_consensus.cc:729] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43d727296b1b416c997624f54525d7e1, State: Initialized, Role: FOLLOWER
I20260925 09:31:21.121516 28863 consensus_queue.cc:260] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3663102c98748d88b2218aa1768e96c" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41355 } } peers { permanent_uuid: "43d727296b1b416c997624f54525d7e1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44741 } } peers { permanent_uuid: "445a59abeb8f4c8db0a3b4b8a330b6e4" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34015 } }
I20260925 09:31:21.121889 28862 ts_tablet_manager.cc:1428] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4: Time spent starting tablet: real 0.017s user 0.014s sys 0.002s
I20260925 09:31:21.122421 28865 raft_consensus.cc:348] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3663102c98748d88b2218aa1768e96c" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41355 } } peers { permanent_uuid: "43d727296b1b416c997624f54525d7e1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44741 } } peers { permanent_uuid: "445a59abeb8f4c8db0a3b4b8a330b6e4" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34015 } }
I20260925 09:31:21.122774 28865 raft_consensus.cc:374] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:21.122897 28865 raft_consensus.cc:729] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3663102c98748d88b2218aa1768e96c, State: Initialized, Role: FOLLOWER
I20260925 09:31:21.123513 28865 consensus_queue.cc:260] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3663102c98748d88b2218aa1768e96c" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41355 } } peers { permanent_uuid: "43d727296b1b416c997624f54525d7e1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44741 } } peers { permanent_uuid: "445a59abeb8f4c8db0a3b4b8a330b6e4" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34015 } }
I20260925 09:31:21.125623 28865 ts_tablet_manager.cc:1428] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260925 09:31:21.126101 28714 heartbeater.cc:503] Master 127.10.114.126:40645 was elected leader, sending a full tablet report...
I20260925 09:31:21.126787 28844 heartbeater.cc:503] Master 127.10.114.126:40645 was elected leader, sending a full tablet report...
I20260925 09:31:21.127415 28863 ts_tablet_manager.cc:1428] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1: Time spent starting tablet: real 0.016s user 0.007s sys 0.008s
W20260925 09:31:21.235471 28845 tablet.cc:2369] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:31:21.239135 28870 raft_consensus.cc:482] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:31:21.239462 28870 raft_consensus.cc:504] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3663102c98748d88b2218aa1768e96c" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41355 } } peers { permanent_uuid: "43d727296b1b416c997624f54525d7e1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44741 } } peers { permanent_uuid: "445a59abeb8f4c8db0a3b4b8a330b6e4" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34015 } }
I20260925 09:31:21.241241 28870 leader_election.cc:288] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43d727296b1b416c997624f54525d7e1 (127.10.114.67:44741), 445a59abeb8f4c8db0a3b4b8a330b6e4 (127.10.114.65:34015)
I20260925 09:31:21.248184 28872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:21.241551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:34015 (local address 127.10.114.66:52979)
0925 09:31:21.242482 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:21.242521 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:21.242704 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:21.243107 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:21.243110 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:21.243140 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:21.243505 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:21.243513 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.244637 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.244643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:21.246228 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:21.246239 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.247428 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.247434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:21.247625 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:21.247650 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 09:31:21.247723 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 09:31:21.247890 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"thread_start_us":707,"threads_started":1}
I20260925 09:31:21.248216 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:21.241507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:53925 (local address 127.10.114.67:44741)
0925 09:31:21.241721 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:21.241730 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:21.242542 (+ 812us) server_negotiation.cc:407] Connection header received
0925 09:31:21.242783 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:21.242788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:21.242865 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:21.242929 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:21.243641 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.244633 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.246587 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.247315 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.247487 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:21.247556 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:21.247903 (+ 347us) server_negotiation.cc:299] Negotiation successful
0925 09:31:21.248017 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:31:21.248235 28860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:21.241794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:52979 (local address 127.10.114.65:34015)
0925 09:31:21.241998 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:21.242005 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:21.242646 (+ 641us) server_negotiation.cc:407] Connection header received
0925 09:31:21.242860 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:21.242865 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:21.242979 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:21.243088 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:21.243700 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.244507 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.246413 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.247232 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.247377 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:21.247485 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:21.247812 (+ 327us) server_negotiation.cc:299] Negotiation successful
0925 09:31:21.247956 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 09:31:21.248327 28871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:21.241427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:44741 (local address 127.10.114.66:53925)
0925 09:31:21.242402 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:21.242425 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:21.242599 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:21.242982 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:21.242988 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:21.243016 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:21.243493 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:21.243503 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.244821 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.244828 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:21.246403 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:21.246417 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.247548 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.247557 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:21.247695 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:21.247754 (+ 59us) client_negotiation.cc:769] Sending connection context
0925 09:31:21.247878 (+ 124us) client_negotiation.cc:240] Negotiation successful
0925 09:31:21.247984 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"mutex_wait_us":3,"thread_start_us":498,"threads_started":1}
I20260925 09:31:21.249393 28538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "244049c7028a4321bc726b99b72f27da" candidate_uuid: "e3663102c98748d88b2218aa1768e96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445a59abeb8f4c8db0a3b4b8a330b6e4" is_pre_election: true
I20260925 09:31:21.249512 28799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "244049c7028a4321bc726b99b72f27da" candidate_uuid: "e3663102c98748d88b2218aa1768e96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43d727296b1b416c997624f54525d7e1" is_pre_election: true
I20260925 09:31:21.249897 28538 raft_consensus.cc:2454] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3663102c98748d88b2218aa1768e96c in term 0.
I20260925 09:31:21.250068 28799 raft_consensus.cc:2454] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3663102c98748d88b2218aa1768e96c in term 0.
I20260925 09:31:21.250535 28604 leader_election.cc:302] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 445a59abeb8f4c8db0a3b4b8a330b6e4, e3663102c98748d88b2218aa1768e96c; no voters:
I20260925 09:31:21.250887 28870 raft_consensus.cc:2781] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:31:21.251024 28870 raft_consensus.cc:482] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:31:21.251125 28870 raft_consensus.cc:3037] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:21.253306 28870 raft_consensus.cc:504] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3663102c98748d88b2218aa1768e96c" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41355 } } peers { permanent_uuid: "43d727296b1b416c997624f54525d7e1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44741 } } peers { permanent_uuid: "445a59abeb8f4c8db0a3b4b8a330b6e4" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34015 } }
I20260925 09:31:21.253868 28870 leader_election.cc:288] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [CANDIDATE]: Term 1 election: Requested vote from peers 43d727296b1b416c997624f54525d7e1 (127.10.114.67:44741), 445a59abeb8f4c8db0a3b4b8a330b6e4 (127.10.114.65:34015)
I20260925 09:31:21.254266 28799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "244049c7028a4321bc726b99b72f27da" candidate_uuid: "e3663102c98748d88b2218aa1768e96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43d727296b1b416c997624f54525d7e1"
I20260925 09:31:21.254304 28538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "244049c7028a4321bc726b99b72f27da" candidate_uuid: "e3663102c98748d88b2218aa1768e96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445a59abeb8f4c8db0a3b4b8a330b6e4"
I20260925 09:31:21.254475 28799 raft_consensus.cc:3037] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:21.254477 28538 raft_consensus.cc:3037] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:21.256709 28538 raft_consensus.cc:2454] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3663102c98748d88b2218aa1768e96c in term 1.
I20260925 09:31:21.256882 28799 raft_consensus.cc:2454] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3663102c98748d88b2218aa1768e96c in term 1.
I20260925 09:31:21.257109 28604 leader_election.cc:302] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 445a59abeb8f4c8db0a3b4b8a330b6e4, e3663102c98748d88b2218aa1768e96c; no voters:
I20260925 09:31:21.257385 28870 raft_consensus.cc:2781] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:21.257874 28870 raft_consensus.cc:686] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [term 1 LEADER]: Becoming Leader. State: Replica: e3663102c98748d88b2218aa1768e96c, State: Running, Role: LEADER
I20260925 09:31:21.258339 28870 consensus_queue.cc:237] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3663102c98748d88b2218aa1768e96c" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41355 } } peers { permanent_uuid: "43d727296b1b416c997624f54525d7e1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44741 } } peers { permanent_uuid: "445a59abeb8f4c8db0a3b4b8a330b6e4" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34015 } }
I20260925 09:31:21.263096 28396 catalog_manager.cc:5495] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c reported cstate change: term changed from 0 to 1, leader changed from <none> to e3663102c98748d88b2218aa1768e96c (127.10.114.66). New cstate: current_term: 1 leader_uuid: "e3663102c98748d88b2218aa1768e96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3663102c98748d88b2218aa1768e96c" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43d727296b1b416c997624f54525d7e1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "445a59abeb8f4c8db0a3b4b8a330b6e4" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34015 } health_report { overall_health: UNKNOWN } } }
W20260925 09:31:21.289970 28715 tablet.cc:2369] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:31:21.294517 28858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:21.289308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46538 (local address 127.10.114.66:41355)
0925 09:31:21.289492 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:21.289499 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:21.289522 (+ 23us) server_negotiation.cc:407] Connection header received
0925 09:31:21.289614 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:21.289618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:21.289698 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:21.289891 (+ 193us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:31:21.290360 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.291154 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.292601 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.293006 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.293108 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:21.293521 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:31:21.293956 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:31:21.294063 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:21.294158 (+ 95us) server_negotiation.cc:299] Negotiation successful
0925 09:31:21.294236 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 09:31:21.301982 28799 raft_consensus.cc:1261] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1 [term 1 FOLLOWER]: Refusing update from remote peer e3663102c98748d88b2218aa1768e96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:31:21.301983 28538 raft_consensus.cc:1261] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4 [term 1 FOLLOWER]: Refusing update from remote peer e3663102c98748d88b2218aa1768e96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:31:21.302896 28873 consensus_queue.cc:1035] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "445a59abeb8f4c8db0a3b4b8a330b6e4" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:31:21.303192 28870 consensus_queue.cc:1035] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43d727296b1b416c997624f54525d7e1" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:31:21.316048 28878 mvcc.cc:205] Tried to move back new op lower bound from 7333186278603366400 to 7333186278438002688. Current Snapshot: MvccSnapshot[applied={T|T < 7333186278603366400}]
I20260925 09:31:21.337597 28396 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:31:21.337894 28396 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:31:21.341395 28396 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f2123cdea3e448a8b22688d2ab25020: Sending DeleteTablet for 3 replicas of tablet 244049c7028a4321bc726b99b72f27da
I20260925 09:31:21.342465 28779 tablet_service.cc:1514] Processing DeleteTablet for tablet 244049c7028a4321bc726b99b72f27da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:31:21 UTC) from {username='slave'} at 127.0.0.1:42646
I20260925 09:31:21.342556 28518 tablet_service.cc:1514] Processing DeleteTablet for tablet 244049c7028a4321bc726b99b72f27da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:31:21 UTC) from {username='slave'} at 127.0.0.1:41728
I20260925 09:31:21.343245 28649 tablet_service.cc:1514] Processing DeleteTablet for tablet 244049c7028a4321bc726b99b72f27da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:31:21 UTC) from {username='slave'} at 127.0.0.1:46532
I20260925 09:31:21.343384 28886 tablet_replica.cc:330] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1: stopping tablet replica
I20260925 09:31:21.343729 28887 tablet_replica.cc:330] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4: stopping tablet replica
I20260925 09:31:21.343803 28886 raft_consensus.cc:2229] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:31:21.344053 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 28456
I20260925 09:31:21.344041 28888 tablet_replica.cc:330] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c: stopping tablet replica
I20260925 09:31:21.344126 28887 raft_consensus.cc:2229] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:31:21.344288 28886 raft_consensus.cc:2258] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:31:21.344417 28888 raft_consensus.cc:2229] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:31:21.344594 28887 raft_consensus.cc:2258] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:31:21.345487 28888 raft_consensus.cc:2258] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:31:21.346424 28886 ts_tablet_manager.cc:1905] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:31:21.346545 28887 ts_tablet_manager.cc:1905] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:31:21.348682 28888 ts_tablet_manager.cc:1905] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:31:21.353199 28886 ts_tablet_manager.cc:1918] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:31:21.353358 28887 ts_tablet_manager.cc:1918] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:31:21.353398 28886 log.cc:1196] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/244049c7028a4321bc726b99b72f27da
I20260925 09:31:21.353623 28887 log.cc:1196] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/244049c7028a4321bc726b99b72f27da
I20260925 09:31:21.353904 28886 ts_tablet_manager.cc:1939] T 244049c7028a4321bc726b99b72f27da P 43d727296b1b416c997624f54525d7e1: Deleting consensus metadata
I20260925 09:31:21.354244 28887 ts_tablet_manager.cc:1939] T 244049c7028a4321bc726b99b72f27da P 445a59abeb8f4c8db0a3b4b8a330b6e4: Deleting consensus metadata
I20260925 09:31:21.354836 28860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:21.346750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41738 (local address 127.10.114.65:34015)
0925 09:31:21.346926 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:21.346932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:21.346952 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:31:21.347005 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:21.347012 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:21.347107 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:21.347214 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:21.347814 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.349029 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.350905 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.351594 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.351702 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:21.353139 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:21.353152 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:21.353169 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:21.353918 (+ 749us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:21.353976 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:21.354007 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:21.354094 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:21.354358 (+ 264us) server_negotiation.cc:299] Negotiation successful
0925 09:31:21.354471 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:31:21.355616 28381 catalog_manager.cc:4841] TS 43d727296b1b416c997624f54525d7e1 (127.10.114.67:44741): tablet 244049c7028a4321bc726b99b72f27da (table test-table [id=d9767c4c33134a48aa8cfb1288cdd11c]) successfully deleted
I20260925 09:31:21.355980 28383 catalog_manager.cc:4841] TS 445a59abeb8f4c8db0a3b4b8a330b6e4 (127.10.114.65:34015): tablet 244049c7028a4321bc726b99b72f27da (table test-table [id=d9767c4c33134a48aa8cfb1288cdd11c]) successfully deleted
I20260925 09:31:21.356594 28578 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:21.358179 28888 ts_tablet_manager.cc:1918] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:31:21.358353 28888 log.cc:1196] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/244049c7028a4321bc726b99b72f27da
I20260925 09:31:21.358793 28888 ts_tablet_manager.cc:1939] T 244049c7028a4321bc726b99b72f27da P e3663102c98748d88b2218aa1768e96c: Deleting consensus metadata
I20260925 09:31:21.359998 28383 catalog_manager.cc:4841] TS e3663102c98748d88b2218aa1768e96c (127.10.114.66:41355): tablet 244049c7028a4321bc726b99b72f27da (table test-table [id=d9767c4c33134a48aa8cfb1288cdd11c]) successfully deleted
I20260925 09:31:21.448397 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 28456
I20260925 09:31:21.468345 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 28587
I20260925 09:31:21.476624 28858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:21.469163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46548 (local address 127.10.114.66:41355)
0925 09:31:21.469352 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:21.469358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:21.469381 (+ 23us) server_negotiation.cc:407] Connection header received
0925 09:31:21.469553 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:21.469559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:21.469665 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:21.469774 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:21.470108 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.471257 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.472829 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.473579 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.473740 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:21.474899 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:21.474911 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:21.474929 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:21.475749 (+ 820us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:21.475861 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:21.475886 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:21.475988 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:21.476147 (+ 159us) server_negotiation.cc:299] Negotiation successful
0925 09:31:21.476239 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 09:31:21.477428 28709 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:21.587045 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 28587
I20260925 09:31:21.606349 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 28717
I20260925 09:31:21.613178 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:21.607246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42658 (local address 127.10.114.67:44741)
0925 09:31:21.607401 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:21.607405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:21.607443 (+ 38us) server_negotiation.cc:407] Connection header received
0925 09:31:21.607684 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:21.607688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:21.607834 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:21.607928 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:21.608306 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.609015 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.610375 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:21.610773 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:21.610876 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:21.611974 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:21.611983 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:21.611997 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:21.612538 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:21.612587 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:21.612606 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:21.612692 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:21.612868 (+ 176us) server_negotiation.cc:299] Negotiation successful
0925 09:31:21.612951 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 09:31:21.613783 28839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:21.700450 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 28717
I20260925 09:31:21.717429 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 28366
I20260925 09:31:21.718613 28427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:21.845469 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 28366
2026-09-25T09:31:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2750 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:31:21 dist-test-slave-9pj8 krb5kdc[28896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:31:21 dist-test-slave-9pj8 krb5kdc[28896](info): set up 2 sockets
Sep 25 09:31:21 dist-test-slave-9pj8 krb5kdc[28896](info): commencing operation
krb5kdc: starting...
W20260925 09:31:22.931946 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:31:23 dist-test-slave-9pj8 krb5kdc[28896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:31:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:31:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:31:23.154693 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:42771
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:32861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:42771
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:23.471663 28909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:23.471997 28909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:23.472144 28909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:23.480041 28909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:31:23.480186 28909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:23.480304 28909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:23.480412 28909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:31:23.480494 28909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:31:23.490252 28909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:32861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:42771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:42771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:23.491528 28909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:23.492934 28909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:23.504908 28914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:23.504875 28915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:23.505461 28917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:23.505724 28909 server_base.cc:1034] running on GCE node
I20260925 09:31:23.506420 28909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:23.507676 28909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:23.508909 28909 hybrid_clock.cc:648] HybridClock initialized: now 1790328683508872 us; error 44 us; skew 500 ppm
I20260925 09:31:23.509253 28909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:23 dist-test-slave-9pj8 krb5kdc[28896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328683, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:23.517599 28909 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:31:23.519090 28909 webserver.cc:460] Webserver started at http://127.10.114.126:33719/ using document root <none> and password file <none>
I20260925 09:31:23.519747 28909 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:23.519966 28909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:23.520264 28909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:23.523056 28909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "c35add036a7f44f78286c6c3293d3df7"
format_stamp: "Formatted at 2026-09-25 09:31:23 on dist-test-slave-9pj8"
server_key: "432afc8ddb6cbbe06b1c14a7b5443d4a"
server_key_iv: "aac0081c973383d4bb23f10599265f4b"
server_key_version: "encryptionkey@0"
I20260925 09:31:23.523764 28909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "c35add036a7f44f78286c6c3293d3df7"
format_stamp: "Formatted at 2026-09-25 09:31:23 on dist-test-slave-9pj8"
server_key: "432afc8ddb6cbbe06b1c14a7b5443d4a"
server_key_iv: "aac0081c973383d4bb23f10599265f4b"
server_key_version: "encryptionkey@0"
I20260925 09:31:23.528666 28909 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:31:23.532004 28924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:23.533285 28909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 09:31:23.533461 28909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "c35add036a7f44f78286c6c3293d3df7"
format_stamp: "Formatted at 2026-09-25 09:31:23 on dist-test-slave-9pj8"
server_key: "432afc8ddb6cbbe06b1c14a7b5443d4a"
server_key_iv: "aac0081c973383d4bb23f10599265f4b"
server_key_version: "encryptionkey@0"
I20260925 09:31:23.533653 28909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:23.547736 28909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:23.551445 28909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:23.551712 28909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:23.569528 28909 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:42771
I20260925 09:31:23.569533 28975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:42771 every 8 connection(s)
I20260925 09:31:23.570756 28909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:31:23.574131 28976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:23.574857 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 28909
I20260925 09:31:23.575044 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:31:23.575582 10697 external_mini_cluster.cc:1428] Setting key 6900d6a7f14691ca41363e8d9f6e1760
I20260925 09:31:23.586262 28976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7: Bootstrap starting.
I20260925 09:31:23.589010 28976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7: Neither blocks nor log segments found. Creating new log.
Sep 25 09:31:23 dist-test-slave-9pj8 krb5kdc[28896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:31:23.590253 28976 log.cc:824] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:23.593165 28976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7: No bootstrap required, opened a new log
I20260925 09:31:23.597790 28979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:23.577528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37210 (local address 127.10.114.126:42771)
0925 09:31:23.578240 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:23.578251 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:23.578289 (+ 38us) server_negotiation.cc:407] Connection header received
0925 09:31:23.580374 (+ 2085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:23.580397 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:23.580743 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:23.581121 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:23.582571 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:23.583936 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:23.585213 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:23.585934 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:23.586133 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:23.590507 (+ 4374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:23.590514 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:23.590526 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:23.590560 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:23.593690 (+ 3130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:23.594520 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:23.594526 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:23.594532 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:23.594743 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:23.595136 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:23.595141 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:23.595146 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:23.595631 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:23.596005 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:23.596429 (+ 424us) server_negotiation.cc:299] Negotiation successful
0925 09:31:23.596864 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":518,"thread_start_us":313,"threads_started":1}
I20260925 09:31:23.601285 28976 raft_consensus.cc:348] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35add036a7f44f78286c6c3293d3df7" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42771 } }
I20260925 09:31:23.601614 28976 raft_consensus.cc:374] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:23.601732 28976 raft_consensus.cc:729] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c35add036a7f44f78286c6c3293d3df7, State: Initialized, Role: FOLLOWER
I20260925 09:31:23.602278 28976 consensus_queue.cc:260] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35add036a7f44f78286c6c3293d3df7" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42771 } }
I20260925 09:31:23.602512 28976 raft_consensus.cc:388] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:31:23.602659 28976 raft_consensus.cc:482] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:31:23.602828 28976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:23.604928 28976 raft_consensus.cc:504] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35add036a7f44f78286c6c3293d3df7" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42771 } }
I20260925 09:31:23.605381 28976 leader_election.cc:302] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c35add036a7f44f78286c6c3293d3df7; no voters:
I20260925 09:31:23.605904 28976 leader_election.cc:288] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:31:23.606082 28981 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:23.606767 28981 raft_consensus.cc:686] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7 [term 1 LEADER]: Becoming Leader. State: Replica: c35add036a7f44f78286c6c3293d3df7, State: Running, Role: LEADER
I20260925 09:31:23.607317 28981 consensus_queue.cc:237] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35add036a7f44f78286c6c3293d3df7" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42771 } }
I20260925 09:31:23.608105 28976 sys_catalog.cc:564] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:31:23.610181 28982 sys_catalog.cc:455] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c35add036a7f44f78286c6c3293d3df7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35add036a7f44f78286c6c3293d3df7" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42771 } } }
I20260925 09:31:23.610759 28982 sys_catalog.cc:458] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:23.610328 28983 sys_catalog.cc:455] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c35add036a7f44f78286c6c3293d3df7. Latest consensus state: current_term: 1 leader_uuid: "c35add036a7f44f78286c6c3293d3df7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35add036a7f44f78286c6c3293d3df7" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42771 } } }
I20260925 09:31:23.611613 28988 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:31:23.612648 28983 sys_catalog.cc:458] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:23.616906 28988 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:31:23.625448 28988 catalog_manager.cc:1347] Generated new cluster ID: 073b3593f36647428ac233a60a984f3f
I20260925 09:31:23.625542 28988 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:31:23.646171 28988 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:31:23.648242 28988 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:31:23.659653 28988 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7: Generated new TSK 0
I20260925 09:31:23.660544 28988 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
I20260925 09:31:23.738405 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42771
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:32861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:24.101450 29002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:24.101708 29002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:24.101804 29002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:24.109138 29002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:24.109261 29002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:24.109453 29002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:31:24.119244 29002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:32861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:24.120429 29002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:24.121709 29002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:24.134047 29007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:24.133999 29010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:24.134076 29008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:24.135161 29002 server_base.cc:1034] running on GCE node
I20260925 09:31:24.135687 29002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:24.136545 29002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:24.137732 29002 hybrid_clock.cc:648] HybridClock initialized: now 1790328684137656 us; error 64 us; skew 500 ppm
I20260925 09:31:24.138087 29002 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:24 dist-test-slave-9pj8 krb5kdc[28896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328684, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:24.141163 29002 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:31:24.142556 29002 webserver.cc:460] Webserver started at http://127.10.114.65:36577/ using document root <none> and password file <none>
I20260925 09:31:24.143198 29002 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:24.143333 29002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:24.143643 29002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:24.146361 29002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "c48f968c95994e8ca9bea44742160182"
format_stamp: "Formatted at 2026-09-25 09:31:24 on dist-test-slave-9pj8"
server_key: "9f2b5927adafc413251a7d3e21dbf6b1"
server_key_iv: "2e96cd22be387fc95f9a68c925b359aa"
server_key_version: "encryptionkey@0"
I20260925 09:31:24.147063 29002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "c48f968c95994e8ca9bea44742160182"
format_stamp: "Formatted at 2026-09-25 09:31:24 on dist-test-slave-9pj8"
server_key: "9f2b5927adafc413251a7d3e21dbf6b1"
server_key_iv: "2e96cd22be387fc95f9a68c925b359aa"
server_key_version: "encryptionkey@0"
I20260925 09:31:24.151762 29002 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260925 09:31:24.154992 29017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:24.156435 29002 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:31:24.156629 29002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "c48f968c95994e8ca9bea44742160182"
format_stamp: "Formatted at 2026-09-25 09:31:24 on dist-test-slave-9pj8"
server_key: "9f2b5927adafc413251a7d3e21dbf6b1"
server_key_iv: "2e96cd22be387fc95f9a68c925b359aa"
server_key_version: "encryptionkey@0"
I20260925 09:31:24.156843 29002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:24.173666 29002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:24.177515 29002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:24.177774 29002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:24.178629 29002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:31:24.179961 29002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:31:24.180056 29002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:24.180204 29002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:31:24.180289 29002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:24.212353 29002 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:35605
I20260925 09:31:24.212384 29129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:35605 every 8 connection(s)
I20260925 09:31:24.213673 29002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:31:24.223254 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 29002
I20260925 09:31:24.223457 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:31:24.223980 10697 external_mini_cluster.cc:1428] Setting key b501730d8785ee390f3057140bf1dc9b
Sep 25 09:31:24 dist-test-slave-9pj8 krb5kdc[28896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328684, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:31:24.233111 29132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:24.216129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:59529 (local address 127.10.114.126:42771)
0925 09:31:24.216575 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:24.216580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:24.217410 (+ 830us) server_negotiation.cc:407] Connection header received
0925 09:31:24.219206 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:24.219212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:24.219299 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:24.219415 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:24.220927 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:24.221639 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:24.223357 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:24.223898 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:24.224001 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:24.227599 (+ 3598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:24.227604 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:24.227607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:24.227637 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:24.230153 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:24.230848 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:24.230855 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:24.230858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:24.230980 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:24.231418 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:24.231425 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:24.231428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:24.231695 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:24.231882 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:24.232515 (+ 633us) server_negotiation.cc:299] Negotiation successful
0925 09:31:24.232768 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":254,"threads_started":1}
I20260925 09:31:24.234019 29133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:24.216539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42771 (local address 127.10.114.65:59529)
0925 09:31:24.217259 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:24.217297 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:24.218934 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:24.219654 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:24.219662 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:24.220216 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:31:24.220756 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:24.220773 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:24.221807 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:24.221812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:24.223172 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:24.223187 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:24.224119 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:24.224126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:24.224395 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:24.225082 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:31:24.225103 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:31:24.227383 (+ 2280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:31:24.230297 (+ 2914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:31:24.230305 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:31:24.230319 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:31:24.230732 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:31:24.231107 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:31:24.231112 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:31:24.231115 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:31:24.231309 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:31:24.231846 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:31:24.231853 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:31:24.232169 (+ 316us) client_negotiation.cc:769] Sending connection context
0925 09:31:24.232425 (+ 256us) client_negotiation.cc:240] Negotiation successful
0925 09:31:24.232951 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":297,"threads_started":1}
I20260925 09:31:24.235476 29130 heartbeater.cc:347] Connected to a master server at 127.10.114.126:42771
I20260925 09:31:24.235801 29130 heartbeater.cc:464] Registering TS with master...
I20260925 09:31:24.236605 29130 heartbeater.cc:511] Master 127.10.114.126:42771 requested a full tablet report, sending...
I20260925 09:31:24.239104 28941 ts_manager.cc:194] Registered new tserver with Master: c48f968c95994e8ca9bea44742160182 (127.10.114.65:35605)
I20260925 09:31:24.241328 28941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:59529
WARNING: no policy specified for kudu/127.10.114.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
I20260925 09:31:24.277490 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42771
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:32861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:24.589223 29136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:24.589452 29136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:24.589603 29136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:24.596343 29136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:24.596431 29136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:24.596630 29136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:31:24.605579 29136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:32861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:24.606738 29136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:24.608002 29136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:24.620486 29141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:24.620577 29142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:24.620596 29144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:24.621346 29136 server_base.cc:1034] running on GCE node
I20260925 09:31:24.621915 29136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:24.622879 29136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:24.624101 29136 hybrid_clock.cc:648] HybridClock initialized: now 1790328684624054 us; error 64 us; skew 500 ppm
I20260925 09:31:24.624459 29136 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:24 dist-test-slave-9pj8 krb5kdc[28896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328684, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:24.627453 29136 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:31:24.628968 29136 webserver.cc:460] Webserver started at http://127.10.114.66:38905/ using document root <none> and password file <none>
I20260925 09:31:24.629648 29136 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:24.629783 29136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:24.630072 29136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:24.632863 29136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "743580e02ddc47c4acddfa655c7cb5bf"
format_stamp: "Formatted at 2026-09-25 09:31:24 on dist-test-slave-9pj8"
server_key: "75cf5fb7b4db63f0069294b967667a90"
server_key_iv: "79934acb4e60a2166db18ea2374d52e3"
server_key_version: "encryptionkey@0"
I20260925 09:31:24.633596 29136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "743580e02ddc47c4acddfa655c7cb5bf"
format_stamp: "Formatted at 2026-09-25 09:31:24 on dist-test-slave-9pj8"
server_key: "75cf5fb7b4db63f0069294b967667a90"
server_key_iv: "79934acb4e60a2166db18ea2374d52e3"
server_key_version: "encryptionkey@0"
I20260925 09:31:24.638545 29136 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260925 09:31:24.641949 29151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:24.643254 29136 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:31:24.643440 29136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "743580e02ddc47c4acddfa655c7cb5bf"
format_stamp: "Formatted at 2026-09-25 09:31:24 on dist-test-slave-9pj8"
server_key: "75cf5fb7b4db63f0069294b967667a90"
server_key_iv: "79934acb4e60a2166db18ea2374d52e3"
server_key_version: "encryptionkey@0"
I20260925 09:31:24.643640 29136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:24.657100 29136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:24.661621 29136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:24.661944 29136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:24.662936 29136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:31:24.664338 29136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:31:24.664448 29136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:24.664603 29136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:31:24.664687 29136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:24.699275 29136 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:42505
I20260925 09:31:24.699328 29263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:42505 every 8 connection(s)
I20260925 09:31:24.700654 29136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:31:24.707379 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 29136
I20260925 09:31:24.707564 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:31:24.708122 10697 external_mini_cluster.cc:1428] Setting key 5fe5759d9ef149da2cb8be934d4c50ba
Sep 25 09:31:24 dist-test-slave-9pj8 krb5kdc[28896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328684, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:31:24.721931 29132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:24.703181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:33741 (local address 127.10.114.126:42771)
0925 09:31:24.703394 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:24.703398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:24.704596 (+ 1198us) server_negotiation.cc:407] Connection header received
0925 09:31:24.706335 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:24.706340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:24.706431 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:24.706537 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:24.708391 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:24.709228 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:24.711503 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:24.711990 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:24.712089 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:24.716362 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:24.716367 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:24.716370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:24.716400 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:24.718956 (+ 2556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:24.719660 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:24.719669 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:24.719671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:24.719845 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:24.720256 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:24.720260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:24.720263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:24.720579 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:24.720730 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:24.721367 (+ 637us) server_negotiation.cc:299] Negotiation successful
0925 09:31:24.721602 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 09:31:24.722811 29266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:24.703679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42771 (local address 127.10.114.66:33741)
0925 09:31:24.704446 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:24.704487 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:24.706079 (+ 1592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:24.706759 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:24.706766 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:24.707408 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:31:24.708192 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:24.708216 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:24.709430 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:24.709434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:24.711344 (+ 1910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:24.711362 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:24.712215 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:24.712226 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:24.712550 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:24.713282 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:31:24.713317 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:31:24.716146 (+ 2829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:31:24.719131 (+ 2985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:31:24.719139 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:31:24.719153 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:31:24.719536 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:31:24.719947 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:31:24.719952 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:31:24.719956 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:31:24.720134 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:31:24.720700 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:31:24.720707 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:31:24.721020 (+ 313us) client_negotiation.cc:769] Sending connection context
0925 09:31:24.721260 (+ 240us) client_negotiation.cc:240] Negotiation successful
0925 09:31:24.721768 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":287,"threads_started":1}
I20260925 09:31:24.724327 29264 heartbeater.cc:347] Connected to a master server at 127.10.114.126:42771
I20260925 09:31:24.724678 29264 heartbeater.cc:464] Registering TS with master...
I20260925 09:31:24.725567 29264 heartbeater.cc:511] Master 127.10.114.126:42771 requested a full tablet report, sending...
I20260925 09:31:24.727304 28941 ts_manager.cc:194] Registered new tserver with Master: 743580e02ddc47c4acddfa655c7cb5bf (127.10.114.66:42505)
I20260925 09:31:24.728601 28941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:33741
WARNING: no policy specified for kudu/127.10.114.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
I20260925 09:31:24.763387 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42771
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:32861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:25.076467 29269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:25.076740 29269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:25.076924 29269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:25.083911 29269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:25.084035 29269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:25.084195 29269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:31:25.093209 29269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:32861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:42771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:25.094395 29269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:25.095666 29269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:25.108155 29274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:25.108136 29277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:25.108191 29275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:25.109413 29269 server_base.cc:1034] running on GCE node
I20260925 09:31:25.109997 29269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:25.110890 29269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:25.112118 29269 hybrid_clock.cc:648] HybridClock initialized: now 1790328685112081 us; error 58 us; skew 500 ppm
I20260925 09:31:25.112442 29269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:25 dist-test-slave-9pj8 krb5kdc[28896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328685, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:25.115671 29269 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:31:25.117121 29269 webserver.cc:460] Webserver started at http://127.10.114.67:35769/ using document root <none> and password file <none>
I20260925 09:31:25.117779 29269 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:25.117931 29269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:25.118222 29269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:25.121047 29269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "88ac84a64e1748ee8f3a9931d9d61746"
format_stamp: "Formatted at 2026-09-25 09:31:25 on dist-test-slave-9pj8"
server_key: "2bfaaac9ef06df5c704f6180483edec4"
server_key_iv: "1193a26ad4b1be592b7b1e81bcc89e47"
server_key_version: "encryptionkey@0"
I20260925 09:31:25.121827 29269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "88ac84a64e1748ee8f3a9931d9d61746"
format_stamp: "Formatted at 2026-09-25 09:31:25 on dist-test-slave-9pj8"
server_key: "2bfaaac9ef06df5c704f6180483edec4"
server_key_iv: "1193a26ad4b1be592b7b1e81bcc89e47"
server_key_version: "encryptionkey@0"
I20260925 09:31:25.126652 29269 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260925 09:31:25.129897 29284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:25.131160 29269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 09:31:25.131340 29269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "88ac84a64e1748ee8f3a9931d9d61746"
format_stamp: "Formatted at 2026-09-25 09:31:25 on dist-test-slave-9pj8"
server_key: "2bfaaac9ef06df5c704f6180483edec4"
server_key_iv: "1193a26ad4b1be592b7b1e81bcc89e47"
server_key_version: "encryptionkey@0"
I20260925 09:31:25.131498 29269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:25.154098 29269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:25.158561 29269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:25.158876 29269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:25.160565 29269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:31:25.162249 29269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:31:25.162346 29269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:25.162520 29269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:31:25.162585 29269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:25.196745 29269 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:44311
I20260925 09:31:25.196776 29396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:44311 every 8 connection(s)
I20260925 09:31:25.198096 29269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:31:25.203935 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 29269
I20260925 09:31:25.204124 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:31:25.204676 10697 external_mini_cluster.cc:1428] Setting key 01d080e3c52cf5765a654baa6214f4ee
Sep 25 09:31:25 dist-test-slave-9pj8 krb5kdc[28896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328685, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:31:25.218578 29132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:25.200707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:49757 (local address 127.10.114.126:42771)
0925 09:31:25.200905 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:25.200910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:25.202063 (+ 1153us) server_negotiation.cc:407] Connection header received
0925 09:31:25.204079 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:25.204085 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:25.204172 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:25.204295 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:25.206277 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.207187 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.208964 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.209422 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.209515 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:25.212638 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:25.212643 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:25.212646 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:25.212674 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:25.215436 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:25.216356 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:25.216360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:25.216363 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:25.216482 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:25.216927 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:25.216931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:25.216934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:25.217184 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:25.217336 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:25.218015 (+ 679us) server_negotiation.cc:299] Negotiation successful
0925 09:31:25.218261 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 09:31:25.219551 29399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:25.201140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42771 (local address 127.10.114.67:49757)
0925 09:31:25.201853 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:25.201896 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:25.203716 (+ 1820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:25.204574 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:25.204584 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:25.205215 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:31:25.206012 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:25.206046 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.207369 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.207373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:25.208818 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:25.208829 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.209588 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.209592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:25.209763 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:25.210374 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:31:25.210393 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:31:25.212447 (+ 2054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:31:25.215598 (+ 3151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:31:25.215605 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:31:25.215618 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:31:25.216231 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:31:25.216587 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:31:25.216593 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:31:25.216596 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:31:25.216813 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:31:25.217305 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:31:25.217312 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:31:25.217650 (+ 338us) client_negotiation.cc:769] Sending connection context
0925 09:31:25.217902 (+ 252us) client_negotiation.cc:240] Negotiation successful
0925 09:31:25.218364 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":290,"threads_started":1}
I20260925 09:31:25.220882 29397 heartbeater.cc:347] Connected to a master server at 127.10.114.126:42771
I20260925 09:31:25.221197 29397 heartbeater.cc:464] Registering TS with master...
I20260925 09:31:25.221865 29397 heartbeater.cc:511] Master 127.10.114.126:42771 requested a full tablet report, sending...
I20260925 09:31:25.223292 28941 ts_manager.cc:194] Registered new tserver with Master: 88ac84a64e1748ee8f3a9931d9d61746 (127.10.114.67:44311)
I20260925 09:31:25.224249 28941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:49757
I20260925 09:31:25.231720 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:31:25.244774 29130 heartbeater.cc:503] Master 127.10.114.126:42771 was elected leader, sending a full tablet report...
I20260925 09:31:25.248762 29132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:25.235847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37226 (local address 127.10.114.126:42771)
0925 09:31:25.236042 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:25.236046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:25.236457 (+ 411us) server_negotiation.cc:407] Connection header received
0925 09:31:25.236600 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:25.236603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:25.236655 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:25.236728 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:25.238186 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.238925 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.240327 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.240717 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.240809 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:25.243260 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:25.243265 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:25.243268 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:25.243300 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:25.246052 (+ 2752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:25.246870 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:25.246874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:25.246876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:25.246991 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:25.247365 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:25.247369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:25.247370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:25.247748 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:25.248036 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:25.248400 (+ 364us) server_negotiation.cc:299] Negotiation successful
0925 09:31:25.248531 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 09:31:25.252444 28941 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:31:25.259125 28941 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:31:25.293215 29409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:25.271006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:42505 (local address 127.0.0.1:41494)
0925 09:31:25.284800 (+ 13794us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:25.284819 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:25.284946 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:25.285609 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:25.285614 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:25.285636 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:25.286008 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:25.286018 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.288010 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.288015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:25.290404 (+ 2389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:25.290416 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.292212 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.292218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:25.292431 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:25.292484 (+ 53us) client_negotiation.cc:769] Sending connection context
0925 09:31:25.292561 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 09:31:25.292686 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13665,"mutex_wait_us":4,"thread_start_us":11718,"threads_started":1}
I20260925 09:31:25.293419 29412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:25.271145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36480 (local address 127.10.114.67:44311)
0925 09:31:25.282347 (+ 11202us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:25.282360 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:25.283248 (+ 888us) server_negotiation.cc:407] Connection header received
0925 09:31:25.283702 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:25.283712 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:25.284020 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:25.284172 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:25.285153 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.286615 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.290439 (+ 3824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.291979 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.292103 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:25.292349 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:25.292878 (+ 529us) server_negotiation.cc:299] Negotiation successful
0925 09:31:25.293045 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10909,"thread_start_us":11182,"threads_started":1}
I20260925 09:31:25.293996 29411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:25.271328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41494 (local address 127.10.114.66:42505)
0925 09:31:25.284346 (+ 13018us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:25.284357 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:25.285078 (+ 721us) server_negotiation.cc:407] Connection header received
0925 09:31:25.285171 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:25.285179 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:25.285404 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:25.286193 (+ 789us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:25.286388 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.287835 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.290565 (+ 2730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.292007 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.293190 (+ 1183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:25.293414 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:25.293485 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 09:31:25.293648 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12802,"thread_start_us":13023,"threads_started":1}
I20260925 09:31:25.294344 29410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:25.270977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:44311 (local address 127.0.0.1:36480)
0925 09:31:25.283114 (+ 12137us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:25.283154 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:25.283414 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:25.284481 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:25.284489 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:25.284527 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:25.284968 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:25.284980 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.286814 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.286818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:25.290241 (+ 3423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:25.290255 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.292487 (+ 2232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.292493 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:25.292651 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:25.292682 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 09:31:25.292802 (+ 120us) client_negotiation.cc:240] Negotiation successful
0925 09:31:25.292969 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11888,"mutex_wait_us":107,"thread_start_us":13538,"threads_started":1}
I20260925 09:31:25.297788 29199 tablet_service.cc:1467] Processing CreateTablet for tablet 5760e929dc9b4a66bdccc97bde625901 (DEFAULT_TABLE table=test-table [id=288f37aa20094d79b2666c0bf8b0b1c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:31:25.297704 29332 tablet_service.cc:1467] Processing CreateTablet for tablet 5760e929dc9b4a66bdccc97bde625901 (DEFAULT_TABLE table=test-table [id=288f37aa20094d79b2666c0bf8b0b1c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:31:25.299072 29199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5760e929dc9b4a66bdccc97bde625901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:25.299201 29332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5760e929dc9b4a66bdccc97bde625901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:25.299971 29413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:25.284904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:35605 (local address 127.0.0.1:57828)
0925 09:31:25.291478 (+ 6574us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:25.291501 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:25.291637 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:25.292997 (+ 1360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:25.293001 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:25.293024 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:25.293406 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:25.293417 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.295456 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.295460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:25.297437 (+ 1977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:25.297451 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.299242 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.299248 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:25.299409 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:25.299450 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 09:31:25.299542 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 09:31:25.299660 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6406,"thread_start_us":6857,"threads_started":1}
I20260925 09:31:25.300096 29414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:25.285197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57828 (local address 127.10.114.65:35605)
0925 09:31:25.292416 (+ 7219us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:25.292424 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:25.292449 (+ 25us) server_negotiation.cc:407] Connection header received
0925 09:31:25.292537 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:25.292544 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:25.292801 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:25.293594 (+ 793us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:25.293839 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.295282 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.297604 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.299035 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.299178 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:25.299406 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:25.299613 (+ 207us) server_negotiation.cc:299] Negotiation successful
0925 09:31:25.299795 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7058,"thread_start_us":6877,"threads_started":1}
I20260925 09:31:25.303469 29065 tablet_service.cc:1467] Processing CreateTablet for tablet 5760e929dc9b4a66bdccc97bde625901 (DEFAULT_TABLE table=test-table [id=288f37aa20094d79b2666c0bf8b0b1c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:31:25.304806 29065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5760e929dc9b4a66bdccc97bde625901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:25.311324 29415 tablet_bootstrap.cc:492] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf: Bootstrap starting.
I20260925 09:31:25.313515 29416 tablet_bootstrap.cc:492] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746: Bootstrap starting.
I20260925 09:31:25.314853 29415 tablet_bootstrap.cc:654] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:25.316087 29415 log.cc:824] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:25.317003 29416 tablet_bootstrap.cc:654] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:25.317458 29418 tablet_bootstrap.cc:492] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182: Bootstrap starting.
I20260925 09:31:25.318215 29416 log.cc:824] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:25.319329 29415 tablet_bootstrap.cc:492] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf: No bootstrap required, opened a new log
I20260925 09:31:25.319581 29415 ts_tablet_manager.cc:1397] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260925 09:31:25.321004 29418 tablet_bootstrap.cc:654] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:25.321151 29416 tablet_bootstrap.cc:492] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746: No bootstrap required, opened a new log
I20260925 09:31:25.321390 29416 ts_tablet_manager.cc:1397] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260925 09:31:25.322115 29418 log.cc:824] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:25.324849 29418 tablet_bootstrap.cc:492] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182: No bootstrap required, opened a new log
I20260925 09:31:25.325095 29418 ts_tablet_manager.cc:1397] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260925 09:31:25.327153 29415 raft_consensus.cc:348] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ac84a64e1748ee8f3a9931d9d61746" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44311 } } peers { permanent_uuid: "743580e02ddc47c4acddfa655c7cb5bf" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42505 } } peers { permanent_uuid: "c48f968c95994e8ca9bea44742160182" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35605 } }
I20260925 09:31:25.327486 29415 raft_consensus.cc:374] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:25.327598 29415 raft_consensus.cc:729] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 743580e02ddc47c4acddfa655c7cb5bf, State: Initialized, Role: FOLLOWER
I20260925 09:31:25.328191 29415 consensus_queue.cc:260] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ac84a64e1748ee8f3a9931d9d61746" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44311 } } peers { permanent_uuid: "743580e02ddc47c4acddfa655c7cb5bf" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42505 } } peers { permanent_uuid: "c48f968c95994e8ca9bea44742160182" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35605 } }
I20260925 09:31:25.329252 29264 heartbeater.cc:503] Master 127.10.114.126:42771 was elected leader, sending a full tablet report...
I20260925 09:31:25.329849 29415 ts_tablet_manager.cc:1428] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260925 09:31:25.331364 29416 raft_consensus.cc:348] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ac84a64e1748ee8f3a9931d9d61746" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44311 } } peers { permanent_uuid: "743580e02ddc47c4acddfa655c7cb5bf" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42505 } } peers { permanent_uuid: "c48f968c95994e8ca9bea44742160182" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35605 } }
I20260925 09:31:25.331725 29416 raft_consensus.cc:374] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:25.331863 29416 raft_consensus.cc:729] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ac84a64e1748ee8f3a9931d9d61746, State: Initialized, Role: FOLLOWER
I20260925 09:31:25.332403 29416 consensus_queue.cc:260] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ac84a64e1748ee8f3a9931d9d61746" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44311 } } peers { permanent_uuid: "743580e02ddc47c4acddfa655c7cb5bf" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42505 } } peers { permanent_uuid: "c48f968c95994e8ca9bea44742160182" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35605 } }
I20260925 09:31:25.335979 29418 raft_consensus.cc:348] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ac84a64e1748ee8f3a9931d9d61746" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44311 } } peers { permanent_uuid: "743580e02ddc47c4acddfa655c7cb5bf" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42505 } } peers { permanent_uuid: "c48f968c95994e8ca9bea44742160182" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35605 } }
I20260925 09:31:25.336357 29397 heartbeater.cc:503] Master 127.10.114.126:42771 was elected leader, sending a full tablet report...
I20260925 09:31:25.336416 29418 raft_consensus.cc:374] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:25.336555 29418 raft_consensus.cc:729] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48f968c95994e8ca9bea44742160182, State: Initialized, Role: FOLLOWER
I20260925 09:31:25.337081 29416 ts_tablet_manager.cc:1428] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746: Time spent starting tablet: real 0.016s user 0.013s sys 0.003s
I20260925 09:31:25.337097 29418 consensus_queue.cc:260] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ac84a64e1748ee8f3a9931d9d61746" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44311 } } peers { permanent_uuid: "743580e02ddc47c4acddfa655c7cb5bf" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42505 } } peers { permanent_uuid: "c48f968c95994e8ca9bea44742160182" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35605 } }
I20260925 09:31:25.338368 29418 ts_tablet_manager.cc:1428] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
W20260925 09:31:25.449833 29398 tablet.cc:2369] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:31:25.452639 29265 tablet.cc:2369] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:31:25.466079 29131 tablet.cc:2369] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:31:25.526973 29421 raft_consensus.cc:482] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:31:25.527343 29421 raft_consensus.cc:504] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ac84a64e1748ee8f3a9931d9d61746" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44311 } } peers { permanent_uuid: "743580e02ddc47c4acddfa655c7cb5bf" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42505 } } peers { permanent_uuid: "c48f968c95994e8ca9bea44742160182" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35605 } }
I20260925 09:31:25.529114 29421 leader_election.cc:288] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ac84a64e1748ee8f3a9931d9d61746 (127.10.114.67:44311), c48f968c95994e8ca9bea44742160182 (127.10.114.65:35605)
I20260925 09:31:25.538445 29414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:25.530657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:35345 (local address 127.10.114.65:35605)
0925 09:31:25.530847 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:25.530851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:25.531025 (+ 174us) server_negotiation.cc:407] Connection header received
0925 09:31:25.531330 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:25.531334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:25.531394 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:25.531472 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:25.532142 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.533144 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.536007 (+ 2863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.537245 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.537415 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:25.537487 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:25.538069 (+ 582us) server_negotiation.cc:299] Negotiation successful
0925 09:31:25.538198 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:31:25.538460 29412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:25.529586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:59627 (local address 127.10.114.67:44311)
0925 09:31:25.529845 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:25.529854 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:25.530403 (+ 549us) server_negotiation.cc:407] Connection header received
0925 09:31:25.530686 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:25.530693 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:25.530791 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:25.530867 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:25.531854 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.532932 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.536007 (+ 3075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.537291 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.537430 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:25.537492 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:25.538075 (+ 583us) server_negotiation.cc:299] Negotiation successful
0925 09:31:25.538198 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:31:25.538872 29424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:25.529586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:44311 (local address 127.10.114.66:59627)
0925 09:31:25.530253 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:25.530275 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:25.530459 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:25.531133 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:25.531140 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:25.531164 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:25.531632 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:25.531650 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.533072 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.533080 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:25.535840 (+ 2760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:25.535857 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.537573 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.537581 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:25.537825 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:25.537937 (+ 112us) client_negotiation.cc:769] Sending connection context
0925 09:31:25.538016 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 09:31:25.538240 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":178,"thread_start_us":341,"threads_started":1}
I20260925 09:31:25.538915 29425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:25.530273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:35605 (local address 127.10.114.66:35345)
0925 09:31:25.530927 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:25.530946 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:25.531133 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:25.531545 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:25.531553 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:25.531588 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:25.531995 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:25.532005 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.533278 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.533286 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:25.535844 (+ 2558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:25.535857 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.537573 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.537581 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:25.537860 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:25.537900 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 09:31:25.538014 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 09:31:25.538238 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":166,"thread_start_us":333,"threads_started":1}
I20260925 09:31:25.540071 29352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5760e929dc9b4a66bdccc97bde625901" candidate_uuid: "743580e02ddc47c4acddfa655c7cb5bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ac84a64e1748ee8f3a9931d9d61746" is_pre_election: true
I20260925 09:31:25.540074 29085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5760e929dc9b4a66bdccc97bde625901" candidate_uuid: "743580e02ddc47c4acddfa655c7cb5bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48f968c95994e8ca9bea44742160182" is_pre_election: true
I20260925 09:31:25.540556 29352 raft_consensus.cc:2454] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 743580e02ddc47c4acddfa655c7cb5bf in term 0.
I20260925 09:31:25.540609 29085 raft_consensus.cc:2454] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 743580e02ddc47c4acddfa655c7cb5bf in term 0.
I20260925 09:31:25.541250 29155 leader_election.cc:302] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 743580e02ddc47c4acddfa655c7cb5bf, 88ac84a64e1748ee8f3a9931d9d61746; no voters:
I20260925 09:31:25.541638 29421 raft_consensus.cc:2781] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:31:25.541816 29421 raft_consensus.cc:482] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:31:25.541946 29421 raft_consensus.cc:3037] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:25.544377 29421 raft_consensus.cc:504] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ac84a64e1748ee8f3a9931d9d61746" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44311 } } peers { permanent_uuid: "743580e02ddc47c4acddfa655c7cb5bf" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42505 } } peers { permanent_uuid: "c48f968c95994e8ca9bea44742160182" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35605 } }
I20260925 09:31:25.545064 29421 leader_election.cc:288] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [CANDIDATE]: Term 1 election: Requested vote from peers 88ac84a64e1748ee8f3a9931d9d61746 (127.10.114.67:44311), c48f968c95994e8ca9bea44742160182 (127.10.114.65:35605)
I20260925 09:31:25.545506 29352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5760e929dc9b4a66bdccc97bde625901" candidate_uuid: "743580e02ddc47c4acddfa655c7cb5bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ac84a64e1748ee8f3a9931d9d61746"
I20260925 09:31:25.545620 29085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5760e929dc9b4a66bdccc97bde625901" candidate_uuid: "743580e02ddc47c4acddfa655c7cb5bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48f968c95994e8ca9bea44742160182"
I20260925 09:31:25.545735 29352 raft_consensus.cc:3037] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:25.545873 29085 raft_consensus.cc:3037] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:25.548148 29352 raft_consensus.cc:2454] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 743580e02ddc47c4acddfa655c7cb5bf in term 1.
I20260925 09:31:25.548575 29155 leader_election.cc:302] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 743580e02ddc47c4acddfa655c7cb5bf, 88ac84a64e1748ee8f3a9931d9d61746; no voters:
I20260925 09:31:25.548732 29085 raft_consensus.cc:2454] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 743580e02ddc47c4acddfa655c7cb5bf in term 1.
I20260925 09:31:25.548875 29421 raft_consensus.cc:2781] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:25.549487 29421 raft_consensus.cc:686] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [term 1 LEADER]: Becoming Leader. State: Replica: 743580e02ddc47c4acddfa655c7cb5bf, State: Running, Role: LEADER
I20260925 09:31:25.550016 29421 consensus_queue.cc:237] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ac84a64e1748ee8f3a9931d9d61746" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44311 } } peers { permanent_uuid: "743580e02ddc47c4acddfa655c7cb5bf" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42505 } } peers { permanent_uuid: "c48f968c95994e8ca9bea44742160182" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35605 } }
I20260925 09:31:25.554730 28940 catalog_manager.cc:5495] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 743580e02ddc47c4acddfa655c7cb5bf (127.10.114.66). New cstate: current_term: 1 leader_uuid: "743580e02ddc47c4acddfa655c7cb5bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ac84a64e1748ee8f3a9931d9d61746" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "743580e02ddc47c4acddfa655c7cb5bf" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c48f968c95994e8ca9bea44742160182" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35605 } health_report { overall_health: UNKNOWN } } }
I20260925 09:31:25.573930 29411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:25.568049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41502 (local address 127.10.114.66:42505)
0925 09:31:25.568230 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:25.568234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:25.568374 (+ 140us) server_negotiation.cc:407] Connection header received
0925 09:31:25.568532 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:25.568535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:25.568602 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:25.568722 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:31:25.569097 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.570195 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.571545 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.572183 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.572313 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:25.572779 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:31:25.573378 (+ 599us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:31:25.573543 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:25.573639 (+ 96us) server_negotiation.cc:299] Negotiation successful
0925 09:31:25.573680 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 09:31:25.582401 29352 raft_consensus.cc:1261] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746 [term 1 FOLLOWER]: Refusing update from remote peer 743580e02ddc47c4acddfa655c7cb5bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:31:25.582465 29085 raft_consensus.cc:1261] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182 [term 1 FOLLOWER]: Refusing update from remote peer 743580e02ddc47c4acddfa655c7cb5bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:31:25.583328 29426 consensus_queue.cc:1035] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ac84a64e1748ee8f3a9931d9d61746" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 44311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:31:25.583715 29421 consensus_queue.cc:1035] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48f968c95994e8ca9bea44742160182" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 35605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:31:25.594514 29429 mvcc.cc:205] Tried to move back new op lower bound from 7333186296135880704 to 7333186296016556032. Current Snapshot: MvccSnapshot[applied={T|T < 7333186296135880704}]
I20260925 09:31:25.608182 28941 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:31:25.608456 28941 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:31:25.611971 28941 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c35add036a7f44f78286c6c3293d3df7: Sending DeleteTablet for 3 replicas of tablet 5760e929dc9b4a66bdccc97bde625901
I20260925 09:31:25.613039 29199 tablet_service.cc:1514] Processing DeleteTablet for tablet 5760e929dc9b4a66bdccc97bde625901 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:31:25 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:41494
I20260925 09:31:25.613165 29332 tablet_service.cc:1514] Processing DeleteTablet for tablet 5760e929dc9b4a66bdccc97bde625901 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:31:25 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:36480
I20260925 09:31:25.613350 29065 tablet_service.cc:1514] Processing DeleteTablet for tablet 5760e929dc9b4a66bdccc97bde625901 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:31:25 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:57828
I20260925 09:31:25.614169 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 29002
I20260925 09:31:25.625248 29439 tablet_replica.cc:330] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746: stopping tablet replica
I20260925 09:31:25.625406 29440 tablet_replica.cc:330] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182: stopping tablet replica
I20260925 09:31:25.625861 29440 raft_consensus.cc:2229] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:31:25.625921 29439 raft_consensus.cc:2229] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:31:25.626376 29440 raft_consensus.cc:2258] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:31:25.626509 29439 raft_consensus.cc:2258] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:31:25.628584 29440 ts_tablet_manager.cc:1905] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:31:25.628836 29439 ts_tablet_manager.cc:1905] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:31:25.635413 29440 ts_tablet_manager.cc:1918] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:31:25.635735 29440 log.cc:1196] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/5760e929dc9b4a66bdccc97bde625901
I20260925 09:31:25.636334 29440 ts_tablet_manager.cc:1939] T 5760e929dc9b4a66bdccc97bde625901 P c48f968c95994e8ca9bea44742160182: Deleting consensus metadata
I20260925 09:31:25.637137 29439 ts_tablet_manager.cc:1918] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:31:25.637456 29439 log.cc:1196] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/5760e929dc9b4a66bdccc97bde625901
I20260925 09:31:25.638063 29439 ts_tablet_manager.cc:1939] T 5760e929dc9b4a66bdccc97bde625901 P 88ac84a64e1748ee8f3a9931d9d61746: Deleting consensus metadata
I20260925 09:31:25.638168 28928 catalog_manager.cc:4841] TS c48f968c95994e8ca9bea44742160182 (127.10.114.65:35605): tablet 5760e929dc9b4a66bdccc97bde625901 (table test-table [id=288f37aa20094d79b2666c0bf8b0b1c1]) successfully deleted
I20260925 09:31:25.638978 29438 tablet_replica.cc:330] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf: stopping tablet replica
I20260925 09:31:25.639422 29438 raft_consensus.cc:2229] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:31:25.640023 28928 catalog_manager.cc:4841] TS 88ac84a64e1748ee8f3a9931d9d61746 (127.10.114.67:44311): tablet 5760e929dc9b4a66bdccc97bde625901 (table test-table [id=288f37aa20094d79b2666c0bf8b0b1c1]) successfully deleted
I20260925 09:31:25.640105 29438 raft_consensus.cc:2258] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:31:25.641774 29438 ts_tablet_manager.cc:1905] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:31:25.647599 29438 ts_tablet_manager.cc:1918] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:31:25.647766 29438 log.cc:1196] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/5760e929dc9b4a66bdccc97bde625901
I20260925 09:31:25.648352 29438 ts_tablet_manager.cc:1939] T 5760e929dc9b4a66bdccc97bde625901 P 743580e02ddc47c4acddfa655c7cb5bf: Deleting consensus metadata
Sep 25 09:31:25 dist-test-slave-9pj8 krb5kdc[28896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:31:25.649789 28925 catalog_manager.cc:4841] TS 743580e02ddc47c4acddfa655c7cb5bf (127.10.114.66:42505): tablet 5760e929dc9b4a66bdccc97bde625901 (table test-table [id=288f37aa20094d79b2666c0bf8b0b1c1]) successfully deleted
I20260925 09:31:25.657338 29414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:25.615498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57844 (local address 127.10.114.65:35605)
0925 09:31:25.615684 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:25.615692 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:25.638845 (+ 23153us) server_negotiation.cc:407] Connection header received
0925 09:31:25.639260 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:25.639265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:25.639368 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:25.639445 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:25.641906 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.642965 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.644841 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.645444 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.645572 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:25.650724 (+ 5152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:25.650740 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:25.650759 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:25.650797 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:25.654373 (+ 3576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:25.655215 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:25.655223 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:25.655227 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:25.655414 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:25.655754 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:25.655762 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:25.655766 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:25.656235 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:25.656557 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:25.656762 (+ 205us) server_negotiation.cc:299] Negotiation successful
0925 09:31:25.656964 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 09:31:25.658028 29125 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:25.763957 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 29002
I20260925 09:31:25.786933 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 29136
Sep 25 09:31:25 dist-test-slave-9pj8 krb5kdc[28896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.66@KRBTEST.COM
I20260925 09:31:25.803763 29411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:25.787809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41508 (local address 127.10.114.66:42505)
0925 09:31:25.787974 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:25.787978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:25.787995 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:31:25.788110 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:25.788114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:25.788194 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:25.788278 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:25.789827 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.790609 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.792402 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.792836 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.792926 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:25.797827 (+ 4901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:25.797839 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:25.797853 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:25.797883 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:25.801030 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:25.801696 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:25.801715 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:25.801717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:25.801854 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:25.802225 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:25.802229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:25.802231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:25.802564 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:25.802940 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:25.803192 (+ 252us) server_negotiation.cc:299] Negotiation successful
0925 09:31:25.803425 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:31:25.804563 29259 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:25.916116 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 29136
I20260925 09:31:25.935822 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 29269
Sep 25 09:31:25 dist-test-slave-9pj8 krb5kdc[28896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.67@KRBTEST.COM
I20260925 09:31:25.950366 29412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:25.936540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36482 (local address 127.10.114.67:44311)
0925 09:31:25.936761 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:25.936767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:25.936795 (+ 28us) server_negotiation.cc:407] Connection header received
0925 09:31:25.936937 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:25.936941 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:25.937056 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:25.937145 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:25.938413 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.939129 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.940384 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:25.940775 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:25.940891 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:25.945158 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:25.945172 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:25.945189 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:25.945221 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:25.948037 (+ 2816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:25.948659 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:25.948665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:25.948668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:25.948812 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:25.949120 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:25.949123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:25.949125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:25.949442 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:25.949655 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:25.949914 (+ 259us) server_negotiation.cc:299] Negotiation successful
0925 09:31:25.950087 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 09:31:25.951002 29392 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:26.037904 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 29269
I20260925 09:31:26.054406 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 28909
I20260925 09:31:26.055493 28971 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:26.162839 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 28909
2026-09-25T09:31:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4317 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-25T09:31:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:31:26Z Disabled control of system clock
I20260925 09:31:26.236431 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:33771
--webserver_interface=127.10.114.126
--webserver_port=0
--builtin_ntp_servers=127.10.114.84:34369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:33771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 09:31:26.559223 29449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:26.559491 29449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:26.559568 29449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:26.566574 29449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:31:26.566666 29449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:26.566704 29449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:26.566752 29449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:31:26.566798 29449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:31:26.575443 29449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:34369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:33771
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:33771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:26.576570 29449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:26.578241 29449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:26.589154 29454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:26.589105 29457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:26.589190 29455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:26.590377 29449 server_base.cc:1034] running on GCE node
I20260925 09:31:26.591085 29449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:26.592425 29449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:26.593618 29449 hybrid_clock.cc:648] HybridClock initialized: now 1790328686593581 us; error 29 us; skew 500 ppm
I20260925 09:31:26.593954 29449 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:31:26.596288 29449 webserver.cc:460] Webserver started at http://127.10.114.126:41251/ using document root <none> and password file <none>
I20260925 09:31:26.596937 29449 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:26.597023 29449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:26.597368 29449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:26.600145 29449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "43730eb73d314ecb83dfbc87d99580e2"
format_stamp: "Formatted at 2026-09-25 09:31:26 on dist-test-slave-9pj8"
server_key: "517e23c805111fbb9894dc7e3e87e095"
server_key_iv: "60c904eae7f9f9c0aa409f780ddc6afc"
server_key_version: "encryptionkey@0"
I20260925 09:31:26.601001 29449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "43730eb73d314ecb83dfbc87d99580e2"
format_stamp: "Formatted at 2026-09-25 09:31:26 on dist-test-slave-9pj8"
server_key: "517e23c805111fbb9894dc7e3e87e095"
server_key_iv: "60c904eae7f9f9c0aa409f780ddc6afc"
server_key_version: "encryptionkey@0"
I20260925 09:31:26.605777 29449 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260925 09:31:26.608989 29463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:26.610306 29449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:31:26.610496 29449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "43730eb73d314ecb83dfbc87d99580e2"
format_stamp: "Formatted at 2026-09-25 09:31:26 on dist-test-slave-9pj8"
server_key: "517e23c805111fbb9894dc7e3e87e095"
server_key_iv: "60c904eae7f9f9c0aa409f780ddc6afc"
server_key_version: "encryptionkey@0"
I20260925 09:31:26.610673 29449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:26.626189 29449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:26.626972 29449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:26.627303 29449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:26.645586 29449 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:33771
I20260925 09:31:26.645670 29514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:33771 every 8 connection(s)
I20260925 09:31:26.646859 29449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:31:26.650628 29515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:26.656513 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 29449
I20260925 09:31:26.656731 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:31:26.657176 10697 external_mini_cluster.cc:1428] Setting key 7b5409e22f3b3591b2bef65414adcabf
I20260925 09:31:26.661926 29515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2: Bootstrap starting.
I20260925 09:31:26.665254 29515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:26.666281 29515 log.cc:824] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:26.669412 29515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2: No bootstrap required, opened a new log
I20260925 09:31:26.670497 29518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:26.659073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37850 (local address 127.10.114.126:33771)
0925 09:31:26.660172 (+ 1099us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:26.660188 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:26.660233 (+ 45us) server_negotiation.cc:407] Connection header received
0925 09:31:26.661852 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:26.661873 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:26.662401 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:26.662778 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:26.663331 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:26.664734 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:26.666118 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:26.666836 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:26.667054 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:26.668280 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:26.668288 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:26.668302 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:26.668815 (+ 513us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:26.668896 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:26.668918 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:26.669021 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:26.669274 (+ 253us) server_negotiation.cc:299] Negotiation successful
0925 09:31:26.669594 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":357,"threads_started":1}
I20260925 09:31:26.677302 29515 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43730eb73d314ecb83dfbc87d99580e2" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33771 } }
I20260925 09:31:26.677644 29515 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:26.677757 29515 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43730eb73d314ecb83dfbc87d99580e2, State: Initialized, Role: FOLLOWER
I20260925 09:31:26.678265 29515 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43730eb73d314ecb83dfbc87d99580e2" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33771 } }
I20260925 09:31:26.678519 29515 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:31:26.678676 29515 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:31:26.678862 29515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:26.681039 29515 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43730eb73d314ecb83dfbc87d99580e2" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33771 } }
I20260925 09:31:26.681502 29515 leader_election.cc:302] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43730eb73d314ecb83dfbc87d99580e2; no voters:
I20260925 09:31:26.682098 29515 leader_election.cc:288] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:31:26.682224 29520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:26.682963 29520 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2 [term 1 LEADER]: Becoming Leader. State: Replica: 43730eb73d314ecb83dfbc87d99580e2, State: Running, Role: LEADER
I20260925 09:31:26.683589 29520 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43730eb73d314ecb83dfbc87d99580e2" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33771 } }
I20260925 09:31:26.684392 29515 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:31:26.686561 29521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43730eb73d314ecb83dfbc87d99580e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43730eb73d314ecb83dfbc87d99580e2" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33771 } } }
I20260925 09:31:26.686656 29522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43730eb73d314ecb83dfbc87d99580e2. Latest consensus state: current_term: 1 leader_uuid: "43730eb73d314ecb83dfbc87d99580e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43730eb73d314ecb83dfbc87d99580e2" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33771 } } }
I20260925 09:31:26.686944 29521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:26.686985 29522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:26.688825 29527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:31:26.694072 29527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:31:26.702487 29527 catalog_manager.cc:1347] Generated new cluster ID: 380cc53a6307414eaea25d5a29192bb8
I20260925 09:31:26.702592 29527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:31:26.715662 29527 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:31:26.716969 29527 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:31:26.723721 29527 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2: Generated new TSK 0
I20260925 09:31:26.724505 29527 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:31:26.729242 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:33771
--builtin_ntp_servers=127.10.114.84:34369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 09:31:27.052676 29539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:27.052937 29539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:27.053099 29539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:27.060058 29539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:27.060153 29539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:27.060354 29539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:31:27.068858 29539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:34369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:33771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:27.070091 29539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:27.071746 29539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:27.084398 29544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:27.084512 29545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:27.084681 29547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:27.085498 29539 server_base.cc:1034] running on GCE node
I20260925 09:31:27.086122 29539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:27.086975 29539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:27.088199 29539 hybrid_clock.cc:648] HybridClock initialized: now 1790328687088163 us; error 35 us; skew 500 ppm
I20260925 09:31:27.088569 29539 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:31:27.090808 29539 webserver.cc:460] Webserver started at http://127.10.114.65:42149/ using document root <none> and password file <none>
I20260925 09:31:27.091501 29539 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:27.091620 29539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:27.092028 29539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:27.094938 29539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "92a2ba77c3b3458b8bde5bdc4dec9da2"
format_stamp: "Formatted at 2026-09-25 09:31:27 on dist-test-slave-9pj8"
server_key: "bd49e47eef96e309588a0de33b0ab1d8"
server_key_iv: "5416317d46793e46c55479e9158b8403"
server_key_version: "encryptionkey@0"
I20260925 09:31:27.095750 29539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "92a2ba77c3b3458b8bde5bdc4dec9da2"
format_stamp: "Formatted at 2026-09-25 09:31:27 on dist-test-slave-9pj8"
server_key: "bd49e47eef96e309588a0de33b0ab1d8"
server_key_iv: "5416317d46793e46c55479e9158b8403"
server_key_version: "encryptionkey@0"
I20260925 09:31:27.100687 29539 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260925 09:31:27.104095 29553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:27.105542 29539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:31:27.105742 29539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "92a2ba77c3b3458b8bde5bdc4dec9da2"
format_stamp: "Formatted at 2026-09-25 09:31:27 on dist-test-slave-9pj8"
server_key: "bd49e47eef96e309588a0de33b0ab1d8"
server_key_iv: "5416317d46793e46c55479e9158b8403"
server_key_version: "encryptionkey@0"
I20260925 09:31:27.105975 29539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:27.118289 29539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:27.119078 29539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:27.119431 29539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:27.120450 29539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:31:27.121809 29539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:31:27.121934 29539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:27.122076 29539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:31:27.122172 29539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:27.157284 29539 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:40611
I20260925 09:31:27.157387 29665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:40611 every 8 connection(s)
I20260925 09:31:27.158670 29539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:31:27.166603 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 29539
I20260925 09:31:27.166770 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:31:27.167225 10697 external_mini_cluster.cc:1428] Setting key 9763ce54c5bcc92372a027c911209bf2
I20260925 09:31:27.169996 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:33771
--builtin_ntp_servers=127.10.114.84:34369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 09:31:27.171629 29518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:27.161306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:55149 (local address 127.10.114.126:33771)
0925 09:31:27.161488 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:27.161492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:27.162763 (+ 1271us) server_negotiation.cc:407] Connection header received
0925 09:31:27.164639 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:27.164644 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:27.164733 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:27.164847 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:27.165906 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:27.166892 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:27.168558 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:27.169057 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:27.169202 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:27.170721 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:27.170730 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:27.170738 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:27.170762 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:27.170805 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:27.170823 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:27.170921 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:27.171114 (+ 193us) server_negotiation.cc:299] Negotiation successful
0925 09:31:27.171222 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:31:27.172359 29668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:27.161773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:33771 (local address 127.10.114.65:55149)
0925 09:31:27.162609 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:27.162642 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:27.164374 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:27.165077 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:27.165084 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:27.165170 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:31:27.165731 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:27.165757 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:27.167038 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:27.167042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:27.168427 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:27.168438 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:27.169313 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:27.169319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:27.169598 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:27.170400 (+ 802us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:31:27.170402 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:31:27.170442 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:31:27.170508 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:31:27.170923 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:31:27.170930 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:31:27.170936 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 09:31:27.171070 (+ 134us) client_negotiation.cc:240] Negotiation successful
0925 09:31:27.171345 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":267,"threads_started":1}
I20260925 09:31:27.173767 29666 heartbeater.cc:347] Connected to a master server at 127.10.114.126:33771
I20260925 09:31:27.174058 29666 heartbeater.cc:464] Registering TS with master...
I20260925 09:31:27.174880 29666 heartbeater.cc:511] Master 127.10.114.126:33771 requested a full tablet report, sending...
I20260925 09:31:27.177016 29480 ts_manager.cc:194] Registered new tserver with Master: 92a2ba77c3b3458b8bde5bdc4dec9da2 (127.10.114.65:40611)
I20260925 09:31:27.178465 29480 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.114.65:55149
W20260925 09:31:27.483999 29669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:27.484246 29669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:27.484387 29669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:27.491392 29669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:27.491475 29669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:27.491575 29669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:31:27.500305 29669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:34369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:33771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:27.501435 29669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:27.503225 29669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:27.516044 29674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:27.516011 29675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:27.516099 29677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:27.516858 29669 server_base.cc:1034] running on GCE node
I20260925 09:31:27.517395 29669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:27.518290 29669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:27.519479 29669 hybrid_clock.cc:648] HybridClock initialized: now 1790328687519423 us; error 47 us; skew 500 ppm
I20260925 09:31:27.519716 29669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:31:27.521842 29669 webserver.cc:460] Webserver started at http://127.10.114.66:43485/ using document root <none> and password file <none>
I20260925 09:31:27.522432 29669 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:27.522514 29669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:27.522763 29669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:27.525439 29669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "cee98107caa84e7397db43a24f377f09"
format_stamp: "Formatted at 2026-09-25 09:31:27 on dist-test-slave-9pj8"
server_key: "9c1ff2121748a2d19e23d4653c937f29"
server_key_iv: "8a866d3291632a0386ce3e048098db34"
server_key_version: "encryptionkey@0"
I20260925 09:31:27.526140 29669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "cee98107caa84e7397db43a24f377f09"
format_stamp: "Formatted at 2026-09-25 09:31:27 on dist-test-slave-9pj8"
server_key: "9c1ff2121748a2d19e23d4653c937f29"
server_key_iv: "8a866d3291632a0386ce3e048098db34"
server_key_version: "encryptionkey@0"
I20260925 09:31:27.530774 29669 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260925 09:31:27.534119 29683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:27.535341 29669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:31:27.535502 29669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "cee98107caa84e7397db43a24f377f09"
format_stamp: "Formatted at 2026-09-25 09:31:27 on dist-test-slave-9pj8"
server_key: "9c1ff2121748a2d19e23d4653c937f29"
server_key_iv: "8a866d3291632a0386ce3e048098db34"
server_key_version: "encryptionkey@0"
I20260925 09:31:27.535687 29669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:27.558560 29669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:27.559320 29669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:27.559608 29669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:27.560559 29669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:31:27.561915 29669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:31:27.562026 29669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:27.562152 29669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:31:27.562247 29669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:27.594839 29669 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:42197
I20260925 09:31:27.594936 29795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:42197 every 8 connection(s)
I20260925 09:31:27.596210 29669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:31:27.604185 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 29669
I20260925 09:31:27.604357 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:31:27.604816 10697 external_mini_cluster.cc:1428] Setting key b635d8383d6288fbb409fe4f16b95503
I20260925 09:31:27.607703 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:33771
--builtin_ntp_servers=127.10.114.84:34369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 09:31:27.609122 29518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:27.598775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:55001 (local address 127.10.114.126:33771)
0925 09:31:27.598962 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:27.598967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:27.600364 (+ 1397us) server_negotiation.cc:407] Connection header received
0925 09:31:27.602131 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:27.602136 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:27.602228 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:27.602337 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:27.603410 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:27.604378 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:27.606124 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:27.606653 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:27.606796 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:27.608183 (+ 1387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:27.608192 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:27.608200 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:27.608225 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:27.608268 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:27.608286 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:27.608386 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:27.608599 (+ 213us) server_negotiation.cc:299] Negotiation successful
0925 09:31:27.608706 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 09:31:27.609925 29798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:27.599313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:33771 (local address 127.10.114.66:55001)
0925 09:31:27.600208 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:27.600243 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:27.601876 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:27.602601 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:27.602609 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:27.602701 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:31:27.603230 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:27.603246 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:27.604545 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:27.604549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:27.605985 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:27.605998 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:27.606883 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:27.606888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:27.607100 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:27.607847 (+ 747us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:31:27.607853 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:31:27.607917 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:31:27.608009 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:31:27.608390 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:31:27.608398 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:31:27.608403 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 09:31:27.608556 (+ 153us) client_negotiation.cc:240] Negotiation successful
0925 09:31:27.608834 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":249,"threads_started":1}
I20260925 09:31:27.611490 29796 heartbeater.cc:347] Connected to a master server at 127.10.114.126:33771
I20260925 09:31:27.611872 29796 heartbeater.cc:464] Registering TS with master...
I20260925 09:31:27.612555 29796 heartbeater.cc:511] Master 127.10.114.126:33771 requested a full tablet report, sending...
I20260925 09:31:27.613952 29480 ts_manager.cc:194] Registered new tserver with Master: cee98107caa84e7397db43a24f377f09 (127.10.114.66:42197)
I20260925 09:31:27.614828 29480 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.114.66:55001
W20260925 09:31:27.926443 29799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:27.926697 29799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:27.926777 29799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:27.933807 29799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:27.933894 29799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:27.934109 29799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:31:27.942864 29799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:34369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:33771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:27.944106 29799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:27.945811 29799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:27.958495 29804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:27.958539 29805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:27.958453 29807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:27.961153 29799 server_base.cc:1034] running on GCE node
I20260925 09:31:27.961728 29799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:27.962519 29799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:27.963737 29799 hybrid_clock.cc:648] HybridClock initialized: now 1790328687963691 us; error 36 us; skew 500 ppm
I20260925 09:31:27.964084 29799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:31:27.966261 29799 webserver.cc:460] Webserver started at http://127.10.114.67:46187/ using document root <none> and password file <none>
I20260925 09:31:27.966869 29799 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:27.966950 29799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:27.967188 29799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:27.969914 29799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "1ba8e83c5e4b42aa988c90a88a9aef92"
format_stamp: "Formatted at 2026-09-25 09:31:27 on dist-test-slave-9pj8"
server_key: "2471a14632055ac1fd81f94d3e3f8016"
server_key_iv: "a22e790d7d4d47d6327d758d98e411b7"
server_key_version: "encryptionkey@0"
I20260925 09:31:27.970618 29799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "1ba8e83c5e4b42aa988c90a88a9aef92"
format_stamp: "Formatted at 2026-09-25 09:31:27 on dist-test-slave-9pj8"
server_key: "2471a14632055ac1fd81f94d3e3f8016"
server_key_iv: "a22e790d7d4d47d6327d758d98e411b7"
server_key_version: "encryptionkey@0"
I20260925 09:31:27.975309 29799 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 09:31:27.978623 29813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:27.979956 29799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:31:27.980114 29799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "1ba8e83c5e4b42aa988c90a88a9aef92"
format_stamp: "Formatted at 2026-09-25 09:31:27 on dist-test-slave-9pj8"
server_key: "2471a14632055ac1fd81f94d3e3f8016"
server_key_iv: "a22e790d7d4d47d6327d758d98e411b7"
server_key_version: "encryptionkey@0"
I20260925 09:31:27.980329 29799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:27.991375 29799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:27.992237 29799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:27.992560 29799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:27.993515 29799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:31:27.994942 29799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:31:27.995121 29799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:27.995288 29799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:31:27.995388 29799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:28.029232 29799 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:43377
I20260925 09:31:28.029286 29925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:43377 every 8 connection(s)
I20260925 09:31:28.030521 29799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:31:28.032563 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 29799
I20260925 09:31:28.032747 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:31:28.033242 10697 external_mini_cluster.cc:1428] Setting key 0e5b8b6c182f70ebd7abd3671415aa3c
I20260925 09:31:28.042956 29518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:28.033204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:57105 (local address 127.10.114.126:33771)
0925 09:31:28.033419 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:28.033425 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:28.034963 (+ 1538us) server_negotiation.cc:407] Connection header received
0925 09:31:28.036766 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:28.036770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:28.036828 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:28.036903 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:28.038015 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.038797 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.040685 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.041057 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.041145 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:28.042365 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:28.042369 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:28.042372 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:28.042386 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:28.042414 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:28.042424 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:28.042486 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:28.042673 (+ 187us) server_negotiation.cc:299] Negotiation successful
0925 09:31:28.042750 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260925 09:31:28.043988 29928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:28.033761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:33771 (local address 127.10.114.67:57105)
0925 09:31:28.034814 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:28.034856 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:28.036538 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:28.037165 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:28.037173 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:28.037289 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:31:28.037856 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:28.037874 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.038972 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.038977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:28.040522 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:28.040534 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.041252 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.041258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:28.041454 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:28.042072 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:31:28.042075 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:31:28.042129 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:31:28.042196 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:31:28.042505 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:31:28.042512 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:31:28.042517 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 09:31:28.042657 (+ 140us) client_negotiation.cc:240] Negotiation successful
0925 09:31:28.042906 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":358,"threads_started":1}
I20260925 09:31:28.045260 29926 heartbeater.cc:347] Connected to a master server at 127.10.114.126:33771
I20260925 09:31:28.045547 29926 heartbeater.cc:464] Registering TS with master...
I20260925 09:31:28.046432 29926 heartbeater.cc:511] Master 127.10.114.126:33771 requested a full tablet report, sending...
I20260925 09:31:28.047973 29480 ts_manager.cc:194] Registered new tserver with Master: 1ba8e83c5e4b42aa988c90a88a9aef92 (127.10.114.67:43377)
I20260925 09:31:28.048764 29480 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.114.67:57105
I20260925 09:31:28.060523 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:31:28.070418 29518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:28.064421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37852 (local address 127.10.114.126:33771)
0925 09:31:28.064658 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:28.064667 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:28.064980 (+ 313us) server_negotiation.cc:407] Connection header received
0925 09:31:28.065098 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:28.065101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:28.065192 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:28.065272 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:28.065704 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.066437 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.067731 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.068159 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.068248 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:28.069415 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:28.069423 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:28.069431 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:28.069455 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:28.069499 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:28.069518 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:28.069698 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:28.069972 (+ 274us) server_negotiation.cc:299] Negotiation successful
0925 09:31:28.070026 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260925 09:31:28.074080 29480 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:31:28.081176 29480 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:31:28.117574 29940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:28.093156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:42197 (local address 127.0.0.1:59836)
0925 09:31:28.104097 (+ 10941us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:28.104148 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:28.104367 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:28.107233 (+ 2866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:28.107244 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:28.107286 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:28.107739 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:28.107754 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.110602 (+ 2848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.110610 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:28.113885 (+ 3275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:28.113912 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.116309 (+ 2397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.116318 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:28.116594 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:28.116666 (+ 72us) client_negotiation.cc:769] Sending connection context
0925 09:31:28.116802 (+ 136us) client_negotiation.cc:240] Negotiation successful
0925 09:31:28.117032 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10584,"mutex_wait_us":98,"thread_start_us":12296,"threads_started":1}
I20260925 09:31:28.118459 29938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:28.093480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:40611 (local address 127.0.0.1:41254)
0925 09:31:28.105800 (+ 12320us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:28.105823 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:28.105954 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:28.108387 (+ 2433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:28.108413 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:28.108451 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:28.108918 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:28.108933 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.111350 (+ 2417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.111358 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:28.114854 (+ 3496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:28.114869 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.117378 (+ 2509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.117388 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:28.117662 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:28.117716 (+ 54us) client_negotiation.cc:769] Sending connection context
0925 09:31:28.117801 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 09:31:28.117974 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12144,"mutex_wait_us":508,"thread_start_us":11099,"threads_started":1}
I20260925 09:31:28.119320 29939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:28.093489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59836 (local address 127.10.114.66:42197)
0925 09:31:28.106456 (+ 12967us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:28.106465 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:28.106495 (+ 30us) server_negotiation.cc:407] Connection header received
0925 09:31:28.106593 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:28.106603 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:28.106943 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:28.108021 (+ 1078us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:28.108274 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.110424 (+ 2150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.114100 (+ 3676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.116060 (+ 1960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.118231 (+ 2171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:28.118546 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:28.118621 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 09:31:28.118838 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12805,"thread_start_us":12709,"threads_started":1}
I20260925 09:31:28.120234 29941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:28.094119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41254 (local address 127.10.114.65:40611)
0925 09:31:28.107720 (+ 13601us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:28.107728 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:28.107755 (+ 27us) server_negotiation.cc:407] Connection header received
0925 09:31:28.107899 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:28.107909 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:28.108164 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:28.108373 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:28.109184 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.111164 (+ 1980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.115103 (+ 3939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.117071 (+ 1968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.119204 (+ 2133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:28.119508 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:28.119603 (+ 95us) server_negotiation.cc:299] Negotiation successful
0925 09:31:28.119847 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13437,"thread_start_us":13318,"threads_started":1}
I20260925 09:31:28.124164 29731 tablet_service.cc:1467] Processing CreateTablet for tablet d6fdf23fc0944fdcaabeccad75bb9233 (DEFAULT_TABLE table=test-table [id=12c337cd415441768e4cc3c67860dc62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:31:28.124164 29601 tablet_service.cc:1467] Processing CreateTablet for tablet d6fdf23fc0944fdcaabeccad75bb9233 (DEFAULT_TABLE table=test-table [id=12c337cd415441768e4cc3c67860dc62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:31:28.125535 29731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fdf23fc0944fdcaabeccad75bb9233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:28.125536 29601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fdf23fc0944fdcaabeccad75bb9233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:28.127830 29940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:28.106522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:43377 (local address 127.0.0.1:34216)
0925 09:31:28.117927 (+ 11405us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:28.117948 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:28.118060 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:28.122294 (+ 4234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:28.122299 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:28.122321 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:28.122523 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:28.122535 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.124397 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.124402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:28.125651 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:28.125692 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.126847 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.126853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:28.127036 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:28.127282 (+ 246us) client_negotiation.cc:769] Sending connection context
0925 09:31:28.127371 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 09:31:28.127501 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11293,"thread_start_us":13438,"threads_started":1}
I20260925 09:31:28.128643 29943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:28.106371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34216 (local address 127.10.114.67:43377)
0925 09:31:28.121841 (+ 15470us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:28.121846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:28.121870 (+ 24us) server_negotiation.cc:407] Connection header received
0925 09:31:28.121935 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:28.121939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:28.122103 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:28.122293 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:28.122701 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.123914 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.125857 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.126690 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.126836 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:28.127085 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:28.128252 (+ 1167us) server_negotiation.cc:299] Negotiation successful
0925 09:31:28.128376 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15345,"thread_start_us":16097,"threads_started":1}
I20260925 09:31:28.131485 29861 tablet_service.cc:1467] Processing CreateTablet for tablet d6fdf23fc0944fdcaabeccad75bb9233 (DEFAULT_TABLE table=test-table [id=12c337cd415441768e4cc3c67860dc62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:31:28.132891 29861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fdf23fc0944fdcaabeccad75bb9233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:28.140645 29945 tablet_bootstrap.cc:492] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2: Bootstrap starting.
I20260925 09:31:28.141041 29944 tablet_bootstrap.cc:492] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09: Bootstrap starting.
I20260925 09:31:28.144840 29944 tablet_bootstrap.cc:654] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:28.145716 29945 tablet_bootstrap.cc:654] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:28.146209 29944 log.cc:824] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:28.146360 29946 tablet_bootstrap.cc:492] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92: Bootstrap starting.
I20260925 09:31:28.147287 29945 log.cc:824] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:28.149415 29944 tablet_bootstrap.cc:492] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09: No bootstrap required, opened a new log
I20260925 09:31:28.149564 29946 tablet_bootstrap.cc:654] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:28.149775 29944 ts_tablet_manager.cc:1397] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260925 09:31:28.150312 29945 tablet_bootstrap.cc:492] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2: No bootstrap required, opened a new log
I20260925 09:31:28.150635 29945 ts_tablet_manager.cc:1397] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260925 09:31:28.150756 29946 log.cc:824] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:28.154215 29946 tablet_bootstrap.cc:492] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92: No bootstrap required, opened a new log
I20260925 09:31:28.154495 29946 ts_tablet_manager.cc:1397] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260925 09:31:28.158303 29944 raft_consensus.cc:348] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cee98107caa84e7397db43a24f377f09" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42197 } } peers { permanent_uuid: "92a2ba77c3b3458b8bde5bdc4dec9da2" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 40611 } } peers { permanent_uuid: "1ba8e83c5e4b42aa988c90a88a9aef92" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 43377 } }
I20260925 09:31:28.158854 29944 raft_consensus.cc:374] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:28.159044 29944 raft_consensus.cc:729] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cee98107caa84e7397db43a24f377f09, State: Initialized, Role: FOLLOWER
I20260925 09:31:28.159902 29944 consensus_queue.cc:260] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cee98107caa84e7397db43a24f377f09" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42197 } } peers { permanent_uuid: "92a2ba77c3b3458b8bde5bdc4dec9da2" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 40611 } } peers { permanent_uuid: "1ba8e83c5e4b42aa988c90a88a9aef92" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 43377 } }
I20260925 09:31:28.163551 29945 raft_consensus.cc:348] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cee98107caa84e7397db43a24f377f09" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42197 } } peers { permanent_uuid: "92a2ba77c3b3458b8bde5bdc4dec9da2" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 40611 } } peers { permanent_uuid: "1ba8e83c5e4b42aa988c90a88a9aef92" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 43377 } }
I20260925 09:31:28.164260 29945 raft_consensus.cc:374] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:28.164444 29945 raft_consensus.cc:729] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92a2ba77c3b3458b8bde5bdc4dec9da2, State: Initialized, Role: FOLLOWER
I20260925 09:31:28.164565 29946 raft_consensus.cc:348] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cee98107caa84e7397db43a24f377f09" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42197 } } peers { permanent_uuid: "92a2ba77c3b3458b8bde5bdc4dec9da2" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 40611 } } peers { permanent_uuid: "1ba8e83c5e4b42aa988c90a88a9aef92" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 43377 } }
I20260925 09:31:28.165007 29946 raft_consensus.cc:374] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:28.165129 29946 raft_consensus.cc:729] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ba8e83c5e4b42aa988c90a88a9aef92, State: Initialized, Role: FOLLOWER
I20260925 09:31:28.165419 29945 consensus_queue.cc:260] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cee98107caa84e7397db43a24f377f09" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42197 } } peers { permanent_uuid: "92a2ba77c3b3458b8bde5bdc4dec9da2" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 40611 } } peers { permanent_uuid: "1ba8e83c5e4b42aa988c90a88a9aef92" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 43377 } }
I20260925 09:31:28.165779 29946 consensus_queue.cc:260] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cee98107caa84e7397db43a24f377f09" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42197 } } peers { permanent_uuid: "92a2ba77c3b3458b8bde5bdc4dec9da2" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 40611 } } peers { permanent_uuid: "1ba8e83c5e4b42aa988c90a88a9aef92" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 43377 } }
I20260925 09:31:28.167683 29796 heartbeater.cc:503] Master 127.10.114.126:33771 was elected leader, sending a full tablet report...
I20260925 09:31:28.168313 29944 ts_tablet_manager.cc:1428] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09: Time spent starting tablet: real 0.018s user 0.010s sys 0.007s
I20260925 09:31:28.173669 29926 heartbeater.cc:503] Master 127.10.114.126:33771 was elected leader, sending a full tablet report...
I20260925 09:31:28.173657 29666 heartbeater.cc:503] Master 127.10.114.126:33771 was elected leader, sending a full tablet report...
I20260925 09:31:28.174034 29946 ts_tablet_manager.cc:1428] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92: Time spent starting tablet: real 0.019s user 0.018s sys 0.000s
I20260925 09:31:28.174322 29945 ts_tablet_manager.cc:1428] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2: Time spent starting tablet: real 0.024s user 0.010s sys 0.011s
I20260925 09:31:28.196920 29950 raft_consensus.cc:482] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:31:28.197199 29950 raft_consensus.cc:504] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cee98107caa84e7397db43a24f377f09" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42197 } } peers { permanent_uuid: "92a2ba77c3b3458b8bde5bdc4dec9da2" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 40611 } } peers { permanent_uuid: "1ba8e83c5e4b42aa988c90a88a9aef92" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 43377 } }
I20260925 09:31:28.198638 29950 leader_election.cc:288] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92a2ba77c3b3458b8bde5bdc4dec9da2 (127.10.114.65:40611), 1ba8e83c5e4b42aa988c90a88a9aef92 (127.10.114.67:43377)
I20260925 09:31:28.205834 29943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:28.199228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:33783 (local address 127.10.114.67:43377)
0925 09:31:28.199480 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:28.199489 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:28.200125 (+ 636us) server_negotiation.cc:407] Connection header received
0925 09:31:28.200304 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:28.200307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:28.200354 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:28.200443 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:28.201033 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.201731 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.204260 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.205010 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.205166 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:28.205232 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:28.205506 (+ 274us) server_negotiation.cc:299] Negotiation successful
0925 09:31:28.205618 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260925 09:31:28.205848 29941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:28.198935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:57655 (local address 127.10.114.65:40611)
0925 09:31:28.199150 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:28.199159 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:28.199990 (+ 831us) server_negotiation.cc:407] Connection header received
0925 09:31:28.200266 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:28.200271 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:28.200356 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:28.200446 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:28.201113 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.201774 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.204207 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.204940 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.205060 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:28.205156 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:28.205492 (+ 336us) server_negotiation.cc:299] Negotiation successful
0925 09:31:28.205610 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260925 09:31:28.205921 29954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:28.199097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:43377 (local address 127.10.114.66:33783)
0925 09:31:28.200022 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:28.200036 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:28.200147 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:28.200459 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:28.200463 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:28.200484 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:28.200875 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:28.200883 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.201871 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.201878 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:28.204044 (+ 2166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:28.204055 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.205199 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.205205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:28.205341 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:28.205363 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 09:31:28.205471 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 09:31:28.205584 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":85,"thread_start_us":617,"threads_started":1}
I20260925 09:31:28.206061 29953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:28.199057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:40611 (local address 127.10.114.66:57655)
0925 09:31:28.199886 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:28.199907 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:28.200058 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:28.200460 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:28.200464 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:28.200586 (+ 122us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:28.200948 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:28.200977 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.201934 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.201937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:28.204011 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:28.204022 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.205135 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.205140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:28.205328 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:28.205351 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 09:31:28.205420 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 09:31:28.205553 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":93,"spinlock_wait_cycles":32896,"thread_start_us":435,"threads_started":1}
I20260925 09:31:28.206905 29881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fdf23fc0944fdcaabeccad75bb9233" candidate_uuid: "cee98107caa84e7397db43a24f377f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ba8e83c5e4b42aa988c90a88a9aef92" is_pre_election: true
I20260925 09:31:28.206899 29621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fdf23fc0944fdcaabeccad75bb9233" candidate_uuid: "cee98107caa84e7397db43a24f377f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92a2ba77c3b3458b8bde5bdc4dec9da2" is_pre_election: true
I20260925 09:31:28.207381 29621 raft_consensus.cc:2454] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cee98107caa84e7397db43a24f377f09 in term 0.
I20260925 09:31:28.207377 29881 raft_consensus.cc:2454] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cee98107caa84e7397db43a24f377f09 in term 0.
I20260925 09:31:28.207989 29684 leader_election.cc:302] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ba8e83c5e4b42aa988c90a88a9aef92, cee98107caa84e7397db43a24f377f09; no voters:
I20260925 09:31:28.208353 29950 raft_consensus.cc:2781] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:31:28.208506 29950 raft_consensus.cc:482] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:31:28.208612 29950 raft_consensus.cc:3037] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:28.210717 29950 raft_consensus.cc:504] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cee98107caa84e7397db43a24f377f09" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42197 } } peers { permanent_uuid: "92a2ba77c3b3458b8bde5bdc4dec9da2" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 40611 } } peers { permanent_uuid: "1ba8e83c5e4b42aa988c90a88a9aef92" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 43377 } }
I20260925 09:31:28.211244 29950 leader_election.cc:288] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [CANDIDATE]: Term 1 election: Requested vote from peers 92a2ba77c3b3458b8bde5bdc4dec9da2 (127.10.114.65:40611), 1ba8e83c5e4b42aa988c90a88a9aef92 (127.10.114.67:43377)
I20260925 09:31:28.211633 29621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fdf23fc0944fdcaabeccad75bb9233" candidate_uuid: "cee98107caa84e7397db43a24f377f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92a2ba77c3b3458b8bde5bdc4dec9da2"
I20260925 09:31:28.211685 29881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fdf23fc0944fdcaabeccad75bb9233" candidate_uuid: "cee98107caa84e7397db43a24f377f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ba8e83c5e4b42aa988c90a88a9aef92"
I20260925 09:31:28.211875 29881 raft_consensus.cc:3037] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:28.211876 29621 raft_consensus.cc:3037] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:28.215273 29621 raft_consensus.cc:2454] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cee98107caa84e7397db43a24f377f09 in term 1.
I20260925 09:31:28.215277 29881 raft_consensus.cc:2454] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cee98107caa84e7397db43a24f377f09 in term 1.
I20260925 09:31:28.215658 29684 leader_election.cc:302] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ba8e83c5e4b42aa988c90a88a9aef92, cee98107caa84e7397db43a24f377f09; no voters:
I20260925 09:31:28.215963 29950 raft_consensus.cc:2781] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:28.216398 29950 raft_consensus.cc:686] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [term 1 LEADER]: Becoming Leader. State: Replica: cee98107caa84e7397db43a24f377f09, State: Running, Role: LEADER
I20260925 09:31:28.216850 29950 consensus_queue.cc:237] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cee98107caa84e7397db43a24f377f09" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42197 } } peers { permanent_uuid: "92a2ba77c3b3458b8bde5bdc4dec9da2" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 40611 } } peers { permanent_uuid: "1ba8e83c5e4b42aa988c90a88a9aef92" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 43377 } }
I20260925 09:31:28.221544 29480 catalog_manager.cc:5495] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 reported cstate change: term changed from 0 to 1, leader changed from <none> to cee98107caa84e7397db43a24f377f09 (127.10.114.66). New cstate: current_term: 1 leader_uuid: "cee98107caa84e7397db43a24f377f09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cee98107caa84e7397db43a24f377f09" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 42197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92a2ba77c3b3458b8bde5bdc4dec9da2" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ba8e83c5e4b42aa988c90a88a9aef92" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 43377 } health_report { overall_health: UNKNOWN } } }
I20260925 09:31:28.257817 29939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:28.253020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59848 (local address 127.10.114.66:42197)
0925 09:31:28.253197 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:28.253202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:28.253347 (+ 145us) server_negotiation.cc:407] Connection header received
0925 09:31:28.253500 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:28.253504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:28.253585 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:28.253708 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:31:28.254021 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.254687 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.255905 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.256311 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.256404 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:28.256868 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:31:28.257358 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:31:28.257495 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:28.257598 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 09:31:28.257628 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 09:31:28.266218 29881 raft_consensus.cc:1261] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92 [term 1 FOLLOWER]: Refusing update from remote peer cee98107caa84e7397db43a24f377f09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:31:28.266285 29621 raft_consensus.cc:1261] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2 [term 1 FOLLOWER]: Refusing update from remote peer cee98107caa84e7397db43a24f377f09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:31:28.267211 29955 consensus_queue.cc:1035] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ba8e83c5e4b42aa988c90a88a9aef92" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:31:28.267513 29950 consensus_queue.cc:1035] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92a2ba77c3b3458b8bde5bdc4dec9da2" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:31:28.277474 29958 mvcc.cc:205] Tried to move back new op lower bound from 7333186307128684544 to 7333186306939928576. Current Snapshot: MvccSnapshot[applied={T|T < 7333186307128684544}]
W20260925 09:31:28.282573 29927 tablet.cc:2369] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:31:28.282807 29959 mvcc.cc:205] Tried to move back new op lower bound from 7333186307128684544 to 7333186306939928576. Current Snapshot: MvccSnapshot[applied={T|T < 7333186307128684544}]
I20260925 09:31:28.293931 29479 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:31:28.294343 29479 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:31:28.298190 29479 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43730eb73d314ecb83dfbc87d99580e2: Sending DeleteTablet for 3 replicas of tablet d6fdf23fc0944fdcaabeccad75bb9233
I20260925 09:31:28.299324 29861 tablet_service.cc:1514] Processing DeleteTablet for tablet d6fdf23fc0944fdcaabeccad75bb9233 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:31:28 UTC) from {username='slave'} at 127.0.0.1:34216
I20260925 09:31:28.299324 29731 tablet_service.cc:1514] Processing DeleteTablet for tablet d6fdf23fc0944fdcaabeccad75bb9233 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:31:28 UTC) from {username='slave'} at 127.0.0.1:59836
I20260925 09:31:28.300002 29601 tablet_service.cc:1514] Processing DeleteTablet for tablet d6fdf23fc0944fdcaabeccad75bb9233 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:31:28 UTC) from {username='slave'} at 127.0.0.1:41254
I20260925 09:31:28.300197 29967 tablet_replica.cc:330] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92: stopping tablet replica
I20260925 09:31:28.300334 29968 tablet_replica.cc:330] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09: stopping tablet replica
I20260925 09:31:28.300586 29967 raft_consensus.cc:2229] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:31:28.300670 29968 raft_consensus.cc:2229] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:31:28.301151 29967 raft_consensus.cc:2258] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:31:28.301388 29968 raft_consensus.cc:2258] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:31:28.302268 29969 tablet_replica.cc:330] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2: stopping tablet replica
I20260925 09:31:28.302691 29969 raft_consensus.cc:2229] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:31:28.302979 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 29539
I20260925 09:31:28.303229 29969 raft_consensus.cc:2258] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:31:28.304833 29967 ts_tablet_manager.cc:1905] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:31:28.305006 29968 ts_tablet_manager.cc:1905] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:31:28.305600 29969 ts_tablet_manager.cc:1905] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:31:28.312047 29967 ts_tablet_manager.cc:1918] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:31:28.312281 29967 log.cc:1196] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/d6fdf23fc0944fdcaabeccad75bb9233
I20260925 09:31:28.312398 29968 ts_tablet_manager.cc:1918] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:31:28.312644 29968 log.cc:1196] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/d6fdf23fc0944fdcaabeccad75bb9233
I20260925 09:31:28.312955 29967 ts_tablet_manager.cc:1939] T d6fdf23fc0944fdcaabeccad75bb9233 P 1ba8e83c5e4b42aa988c90a88a9aef92: Deleting consensus metadata
I20260925 09:31:28.313264 29968 ts_tablet_manager.cc:1939] T d6fdf23fc0944fdcaabeccad75bb9233 P cee98107caa84e7397db43a24f377f09: Deleting consensus metadata
I20260925 09:31:28.314193 29969 ts_tablet_manager.cc:1918] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:31:28.315053 29969 log.cc:1196] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/d6fdf23fc0944fdcaabeccad75bb9233
I20260925 09:31:28.315016 29464 catalog_manager.cc:4841] TS cee98107caa84e7397db43a24f377f09 (127.10.114.66:42197): tablet d6fdf23fc0944fdcaabeccad75bb9233 (table test-table [id=12c337cd415441768e4cc3c67860dc62]) successfully deleted
I20260925 09:31:28.315302 29464 catalog_manager.cc:4841] TS 1ba8e83c5e4b42aa988c90a88a9aef92 (127.10.114.67:43377): tablet d6fdf23fc0944fdcaabeccad75bb9233 (table test-table [id=12c337cd415441768e4cc3c67860dc62]) successfully deleted
I20260925 09:31:28.315748 29969 ts_tablet_manager.cc:1939] T d6fdf23fc0944fdcaabeccad75bb9233 P 92a2ba77c3b3458b8bde5bdc4dec9da2: Deleting consensus metadata
I20260925 09:31:28.317448 29466 catalog_manager.cc:4841] TS 92a2ba77c3b3458b8bde5bdc4dec9da2 (127.10.114.65:40611): tablet d6fdf23fc0944fdcaabeccad75bb9233 (table test-table [id=12c337cd415441768e4cc3c67860dc62]) successfully deleted
I20260925 09:31:28.322597 29941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:28.306293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41258 (local address 127.10.114.65:40611)
0925 09:31:28.306449 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:28.306456 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:28.315375 (+ 8919us) server_negotiation.cc:407] Connection header received
0925 09:31:28.315718 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:28.315727 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:28.315872 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:28.315967 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:28.316645 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.317781 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.319159 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.319842 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.319964 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:28.321038 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:28.321049 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:28.321065 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:28.321851 (+ 786us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:28.321916 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:28.321939 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:28.322049 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:28.322164 (+ 115us) server_negotiation.cc:299] Negotiation successful
0925 09:31:28.322265 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 09:31:28.323356 29661 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:28.411613 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 29539
I20260925 09:31:28.434984 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 29669
I20260925 09:31:28.443079 29939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:28.435969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59854 (local address 127.10.114.66:42197)
0925 09:31:28.436164 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:28.436170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:28.436189 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:31:28.436252 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:28.436257 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:28.436364 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:28.436473 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:28.436816 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.437983 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.439409 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.440276 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.440401 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:28.441389 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:28.441401 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:28.441419 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:28.442251 (+ 832us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:28.442320 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:28.442347 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:28.442452 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:28.442565 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 09:31:28.442687 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 09:31:28.443922 29791 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:28.553193 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 29669
I20260925 09:31:28.571334 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 29799
I20260925 09:31:28.581888 29943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:28.572092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34220 (local address 127.10.114.67:43377)
0925 09:31:28.572263 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:28.572270 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:28.572347 (+ 77us) server_negotiation.cc:407] Connection header received
0925 09:31:28.572549 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:28.572554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:28.572656 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:28.572771 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:28.573155 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.573941 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.575246 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:28.575634 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:28.575767 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:28.580556 (+ 4789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:28.580565 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:28.580580 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:28.581134 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:28.581193 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:28.581215 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:28.581337 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:28.581468 (+ 131us) server_negotiation.cc:299] Negotiation successful
0925 09:31:28.581589 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 09:31:28.582576 29921 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:28.676836 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 29799
I20260925 09:31:28.693686 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 29449
I20260925 09:31:28.694825 29510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:28.802173 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 29449
2026-09-25T09:31:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2639 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:31:28 dist-test-slave-9pj8 krb5kdc[29977](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:31:28 dist-test-slave-9pj8 krb5kdc[29977](info): set up 2 sockets
Sep 25 09:31:28 dist-test-slave-9pj8 krb5kdc[29977](info): commencing operation
krb5kdc: starting...
W20260925 09:31:29.888511 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.006s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:31:30 dist-test-slave-9pj8 krb5kdc[29977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:31:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:31:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:31:30.117264 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:46445
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:42115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:46445
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:30.438427 29990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:30.438697 29990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:30.438876 29990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:30.446061 29990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:31:30.446165 29990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:30.446278 29990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:30.446380 29990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:31:30.446442 29990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:31:30.456545 29990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:42115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:46445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:46445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:30.457818 29990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:30.459144 29990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:30.471360 29995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:30.471323 29996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:30.471671 29998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:30.472023 29990 server_base.cc:1034] running on GCE node
I20260925 09:31:30.472748 29990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:30.474011 29990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:30.475262 29990 hybrid_clock.cc:648] HybridClock initialized: now 1790328690475174 us; error 51 us; skew 500 ppm
I20260925 09:31:30.475561 29990 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:30 dist-test-slave-9pj8 krb5kdc[29977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328690, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:30.478727 29990 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:31:30.480165 29990 webserver.cc:460] Webserver started at http://127.10.114.126:37581/ using document root <none> and password file <none>
I20260925 09:31:30.480798 29990 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:30.480888 29990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:30.481106 29990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:30.483953 29990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "ae4c1151b72247ce809950c30880748c"
format_stamp: "Formatted at 2026-09-25 09:31:30 on dist-test-slave-9pj8"
server_key: "28a332301c034d1ab9351ee2f0452255"
server_key_iv: "d635aef133407c8fe05b734ec63b3d4b"
server_key_version: "encryptionkey@0"
I20260925 09:31:30.484644 29990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "ae4c1151b72247ce809950c30880748c"
format_stamp: "Formatted at 2026-09-25 09:31:30 on dist-test-slave-9pj8"
server_key: "28a332301c034d1ab9351ee2f0452255"
server_key_iv: "d635aef133407c8fe05b734ec63b3d4b"
server_key_version: "encryptionkey@0"
I20260925 09:31:30.489274 29990 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:31:30.492427 30005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:30.493656 29990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:31:30.493825 29990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "ae4c1151b72247ce809950c30880748c"
format_stamp: "Formatted at 2026-09-25 09:31:30 on dist-test-slave-9pj8"
server_key: "28a332301c034d1ab9351ee2f0452255"
server_key_iv: "d635aef133407c8fe05b734ec63b3d4b"
server_key_version: "encryptionkey@0"
I20260925 09:31:30.494002 29990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:30.516503 29990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:30.520429 29990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:30.520714 29990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:30.539065 29990 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:46445
I20260925 09:31:30.539083 30056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:46445 every 8 connection(s)
I20260925 09:31:30.540402 29990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:31:30.544001 30057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:30.547617 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 29990
I20260925 09:31:30.547825 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:31:30.548385 10697 external_mini_cluster.cc:1428] Setting key 0289181a36296730931f34c8da6f087f
I20260925 09:31:30.556566 30057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c: Bootstrap starting.
I20260925 09:31:30.560117 30057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:30.561390 30057 log.cc:824] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c: Log is configured to *not* fsync() on all Append() calls
Sep 25 09:31:30 dist-test-slave-9pj8 krb5kdc[29977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:31:30.564155 30057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c: No bootstrap required, opened a new log
I20260925 09:31:30.570928 30060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:30.550193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33860 (local address 127.10.114.126:46445)
0925 09:31:30.550961 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:30.550971 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:30.551009 (+ 38us) server_negotiation.cc:407] Connection header received
0925 09:31:30.552628 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:30.552648 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:30.553024 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:30.553326 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:30.555051 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:30.556360 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:30.558134 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:30.558752 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:30.558869 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:30.563729 (+ 4860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:30.563736 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:30.563748 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:30.563832 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:30.566916 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:30.567615 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:30.567629 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:30.567634 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:30.567863 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:30.568260 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:30.568265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:30.568268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:30.568777 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:30.569109 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:30.569564 (+ 455us) server_negotiation.cc:299] Negotiation successful
0925 09:31:30.569966 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":362,"threads_started":1}
I20260925 09:31:30.571924 30057 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4c1151b72247ce809950c30880748c" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 46445 } }
I20260925 09:31:30.572280 30057 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:30.572398 30057 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae4c1151b72247ce809950c30880748c, State: Initialized, Role: FOLLOWER
I20260925 09:31:30.572916 30057 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4c1151b72247ce809950c30880748c" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 46445 } }
I20260925 09:31:30.573170 30057 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:31:30.573307 30057 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:31:30.573480 30057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:30.575634 30057 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4c1151b72247ce809950c30880748c" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 46445 } }
I20260925 09:31:30.576102 30057 leader_election.cc:302] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae4c1151b72247ce809950c30880748c; no voters:
I20260925 09:31:30.576627 30057 leader_election.cc:288] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:31:30.576784 30062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:30.577500 30062 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c [term 1 LEADER]: Becoming Leader. State: Replica: ae4c1151b72247ce809950c30880748c, State: Running, Role: LEADER
I20260925 09:31:30.578074 30062 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4c1151b72247ce809950c30880748c" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 46445 } }
I20260925 09:31:30.578639 30057 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:31:30.580742 30064 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae4c1151b72247ce809950c30880748c. Latest consensus state: current_term: 1 leader_uuid: "ae4c1151b72247ce809950c30880748c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4c1151b72247ce809950c30880748c" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 46445 } } }
I20260925 09:31:30.580852 30063 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae4c1151b72247ce809950c30880748c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4c1151b72247ce809950c30880748c" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 46445 } } }
I20260925 09:31:30.581126 30064 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:30.581212 30063 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:30.582103 30068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:31:30.588042 30068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:31:30.596426 30068 catalog_manager.cc:1347] Generated new cluster ID: 7b665db8c4bc4afcba43d2762dea0954
I20260925 09:31:30.596542 30068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:31:30.615437 30068 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:31:30.617267 30068 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:31:30.635143 30068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c: Generated new TSK 0
I20260925 09:31:30.636047 30068 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.114.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
Entry for principal kudu/127.10.114.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab.
I20260925 09:31:30.707178 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:46445
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:42115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:31.029402 30083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:31.029708 30083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:31.029866 30083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:31.036931 30083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:31.037065 30083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:31.037225 30083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:31:31.046465 30083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:42115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.65.keytab
--principal=kudu/127.10.114.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:46445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:31.047634 30083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:31.048925 30083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:31.061784 30091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:31.061884 30088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:31.061863 30089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:31.062132 30083 server_base.cc:1034] running on GCE node
I20260925 09:31:31.063099 30083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:31.063959 30083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:31.065152 30083 hybrid_clock.cc:648] HybridClock initialized: now 1790328691065118 us; error 23 us; skew 500 ppm
I20260925 09:31:31.065434 30083 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:31 dist-test-slave-9pj8 krb5kdc[29977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328691, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:31.068579 30083 init.cc:377] Logged in from keytab as kudu/127.10.114.65@KRBTEST.COM (short username kudu)
I20260925 09:31:31.070070 30083 webserver.cc:460] Webserver started at http://127.10.114.65:43171/ using document root <none> and password file <none>
I20260925 09:31:31.070832 30083 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:31.070961 30083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:31.071334 30083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:31.074383 30083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "2f5e5d6f1be0422dae9e651a00490877"
format_stamp: "Formatted at 2026-09-25 09:31:31 on dist-test-slave-9pj8"
server_key: "e4212b1d75436973aee6045a6f3949be"
server_key_iv: "7d004d9bcfa165e2bb3ae2c5c03104b3"
server_key_version: "encryptionkey@0"
I20260925 09:31:31.075102 30083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "2f5e5d6f1be0422dae9e651a00490877"
format_stamp: "Formatted at 2026-09-25 09:31:31 on dist-test-slave-9pj8"
server_key: "e4212b1d75436973aee6045a6f3949be"
server_key_iv: "7d004d9bcfa165e2bb3ae2c5c03104b3"
server_key_version: "encryptionkey@0"
I20260925 09:31:31.080267 30083 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 09:31:31.083989 30098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:31.085278 30083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:31:31.085450 30083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "2f5e5d6f1be0422dae9e651a00490877"
format_stamp: "Formatted at 2026-09-25 09:31:31 on dist-test-slave-9pj8"
server_key: "e4212b1d75436973aee6045a6f3949be"
server_key_iv: "7d004d9bcfa165e2bb3ae2c5c03104b3"
server_key_version: "encryptionkey@0"
I20260925 09:31:31.085664 30083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:31.101662 30083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:31.105805 30083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:31.106102 30083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:31.107070 30083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:31:31.108502 30083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:31:31.108608 30083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:31.108763 30083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:31:31.108841 30083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:31.143435 30083 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:37387
I20260925 09:31:31.143473 30210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:37387 every 8 connection(s)
I20260925 09:31:31.144824 30083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:31:31.152572 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 30083
I20260925 09:31:31.152760 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:31:31.153294 10697 external_mini_cluster.cc:1428] Setting key ce0b01375f69435984cc2e7045136394
Sep 25 09:31:31 dist-test-slave-9pj8 krb5kdc[29977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328691, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.65@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:31:31.164803 30213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:31.147352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:48923 (local address 127.10.114.126:46445)
0925 09:31:31.147849 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:31.147854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:31.148778 (+ 924us) server_negotiation.cc:407] Connection header received
0925 09:31:31.150507 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:31.150513 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:31.150602 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:31.150747 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:31.152250 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:31.153357 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:31.155009 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:31.155474 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:31.155652 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:31.159121 (+ 3469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:31.159126 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:31.159129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:31.159162 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:31.161788 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:31.162536 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:31.162554 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:31.162559 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:31.162678 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:31.163089 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:31.163099 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:31.163102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:31.163352 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:31.163501 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:31.164235 (+ 734us) server_negotiation.cc:299] Negotiation successful
0925 09:31:31.164465 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":235,"threads_started":1}
I20260925 09:31:31.165649 30214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:31.147897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:46445 (local address 127.10.114.65:48923)
0925 09:31:31.148623 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:31.148665 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:31.150251 (+ 1586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:31.150980 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:31.150988 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:31.151481 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:31:31.152080 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:31.152097 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:31.153531 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:31.153536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:31.154836 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:31.154850 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:31.155745 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:31.155755 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:31.156036 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:31.156730 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:31:31.156753 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:31:31.158891 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:31:31.161978 (+ 3087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:31:31.161986 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:31:31.162001 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:31:31.162409 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:31:31.162791 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:31:31.162796 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:31:31.162799 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:31:31.162973 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:31:31.163476 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:31:31.163483 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:31:31.163878 (+ 395us) client_negotiation.cc:769] Sending connection context
0925 09:31:31.164127 (+ 249us) client_negotiation.cc:240] Negotiation successful
0925 09:31:31.164598 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":268,"threads_started":1}
I20260925 09:31:31.167241 30211 heartbeater.cc:347] Connected to a master server at 127.10.114.126:46445
I20260925 09:31:31.167539 30211 heartbeater.cc:464] Registering TS with master...
I20260925 09:31:31.168377 30211 heartbeater.cc:511] Master 127.10.114.126:46445 requested a full tablet report, sending...
I20260925 09:31:31.170764 30022 ts_manager.cc:194] Registered new tserver with Master: 2f5e5d6f1be0422dae9e651a00490877 (127.10.114.65:37387)
I20260925 09:31:31.172819 30022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.65@KRBTEST.COM'} at 127.10.114.65:48923
WARNING: no policy specified for kudu/127.10.114.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
Entry for principal kudu/127.10.114.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab.
I20260925 09:31:31.208063 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:46445
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:42115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:31.522045 30217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:31.522303 30217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:31.522394 30217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:31.529412 30217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:31.529508 30217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:31.529613 30217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:31:31.538662 30217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:42115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.66.keytab
--principal=kudu/127.10.114.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:46445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:31.539897 30217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:31.541085 30217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:31.553061 30223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:31.553354 30222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:31.554109 30225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:31.554426 30217 server_base.cc:1034] running on GCE node
I20260925 09:31:31.555020 30217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:31.555935 30217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:31.557148 30217 hybrid_clock.cc:648] HybridClock initialized: now 1790328691557110 us; error 47 us; skew 500 ppm
I20260925 09:31:31.557433 30217 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:31 dist-test-slave-9pj8 krb5kdc[29977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328691, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:31.560423 30217 init.cc:377] Logged in from keytab as kudu/127.10.114.66@KRBTEST.COM (short username kudu)
I20260925 09:31:31.561681 30217 webserver.cc:460] Webserver started at http://127.10.114.66:42413/ using document root <none> and password file <none>
I20260925 09:31:31.562300 30217 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:31.562434 30217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:31.562726 30217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:31.565393 30217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "b962fde0d5e143c3af5b9b1ad8bd45cd"
format_stamp: "Formatted at 2026-09-25 09:31:31 on dist-test-slave-9pj8"
server_key: "71f629f32226585fa5f95bde3bc854fa"
server_key_iv: "5f004803c623bd52dc81d06138cdba9f"
server_key_version: "encryptionkey@0"
I20260925 09:31:31.566090 30217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "b962fde0d5e143c3af5b9b1ad8bd45cd"
format_stamp: "Formatted at 2026-09-25 09:31:31 on dist-test-slave-9pj8"
server_key: "71f629f32226585fa5f95bde3bc854fa"
server_key_iv: "5f004803c623bd52dc81d06138cdba9f"
server_key_version: "encryptionkey@0"
I20260925 09:31:31.570981 30217 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260925 09:31:31.574584 30232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:31.575917 30217 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:31:31.576100 30217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "b962fde0d5e143c3af5b9b1ad8bd45cd"
format_stamp: "Formatted at 2026-09-25 09:31:31 on dist-test-slave-9pj8"
server_key: "71f629f32226585fa5f95bde3bc854fa"
server_key_iv: "5f004803c623bd52dc81d06138cdba9f"
server_key_version: "encryptionkey@0"
I20260925 09:31:31.576293 30217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:31.617369 30217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:31.621505 30217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:31.621820 30217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:31.622802 30217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:31:31.624154 30217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:31:31.624256 30217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:31.624405 30217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:31:31.624485 30217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:31.656340 30217 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:41647
I20260925 09:31:31.656417 30344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:41647 every 8 connection(s)
I20260925 09:31:31.657505 30217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:31:31.659955 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 30217
I20260925 09:31:31.660128 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:31:31.660548 10697 external_mini_cluster.cc:1428] Setting key 5bdc03d9080c72758fd371f411e27ed0
Sep 25 09:31:31 dist-test-slave-9pj8 krb5kdc[29977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328691, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.66@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:31:31.677379 30213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:31.659957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:40715 (local address 127.10.114.126:46445)
0925 09:31:31.660176 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:31.660182 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:31.661465 (+ 1283us) server_negotiation.cc:407] Connection header received
0925 09:31:31.663335 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:31.663343 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:31.663463 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:31.663576 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:31.664992 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:31.666081 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:31.667595 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:31.668093 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:31.668229 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:31.671635 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:31.671643 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:31.671647 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:31.671690 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:31.674327 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:31.675190 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:31.675193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:31.675195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:31.675350 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:31.675856 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:31.675860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:31.675861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:31.676113 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:31.676224 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:31.676982 (+ 758us) server_negotiation.cc:299] Negotiation successful
0925 09:31:31.677130 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 09:31:31.678745 30347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:31.660501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:46445 (local address 127.10.114.66:40715)
0925 09:31:31.661309 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:31.661362 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:31.663048 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:31.663756 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:31.663763 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:31.664288 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:31:31.664827 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:31.664843 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:31.666239 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:31.666243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:31.667451 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:31.667463 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:31.668309 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:31.668315 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:31.668530 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:31.669197 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:31:31.669218 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:31:31.671407 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:31:31.674472 (+ 3065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:31:31.674482 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:31:31.674499 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:31:31.675081 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:31:31.675478 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:31:31.675485 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:31:31.675491 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:31:31.675724 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:31:31.676238 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:31:31.676248 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:31:31.676619 (+ 371us) client_negotiation.cc:769] Sending connection context
0925 09:31:31.676903 (+ 284us) client_negotiation.cc:240] Negotiation successful
0925 09:31:31.677418 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":389,"threads_started":1}
I20260925 09:31:31.680301 30345 heartbeater.cc:347] Connected to a master server at 127.10.114.126:46445
I20260925 09:31:31.680593 30345 heartbeater.cc:464] Registering TS with master...
I20260925 09:31:31.681425 30345 heartbeater.cc:511] Master 127.10.114.126:46445 requested a full tablet report, sending...
I20260925 09:31:31.682922 30022 ts_manager.cc:194] Registered new tserver with Master: b962fde0d5e143c3af5b9b1ad8bd45cd (127.10.114.66:41647)
I20260925 09:31:31.684193 30022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.66@KRBTEST.COM'} at 127.10.114.66:40715
WARNING: no policy specified for kudu/127.10.114.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
Entry for principal kudu/127.10.114.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab.
I20260925 09:31:31.715509 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:46445
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:42115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:32.043331 30350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:32.043601 30350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:32.043691 30350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:32.050542 30350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:32.050631 30350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:32.050738 30350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:31:32.059695 30350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:42115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.67.keytab
--principal=kudu/127.10.114.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:46445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:32.060793 30350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:32.061905 30350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:32.074599 30355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:32.075069 30358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:32.074627 30356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:32.075423 30350 server_base.cc:1034] running on GCE node
I20260925 09:31:32.076056 30350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:32.076920 30350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:32.078130 30350 hybrid_clock.cc:648] HybridClock initialized: now 1790328692078100 us; error 54 us; skew 500 ppm
I20260925 09:31:32.078444 30350 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:32 dist-test-slave-9pj8 krb5kdc[29977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328692, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:32.081794 30350 init.cc:377] Logged in from keytab as kudu/127.10.114.67@KRBTEST.COM (short username kudu)
I20260925 09:31:32.083297 30350 webserver.cc:460] Webserver started at http://127.10.114.67:46271/ using document root <none> and password file <none>
I20260925 09:31:32.084034 30350 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:32.084174 30350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:32.084475 30350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:32.087184 30350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "be8db5d114de41639654da2cb561c82e"
format_stamp: "Formatted at 2026-09-25 09:31:32 on dist-test-slave-9pj8"
server_key: "375de12973a193a5b6e40d76e24f2601"
server_key_iv: "3e69ddfdba52e0784a138a50a80e4835"
server_key_version: "encryptionkey@0"
I20260925 09:31:32.088014 30350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "be8db5d114de41639654da2cb561c82e"
format_stamp: "Formatted at 2026-09-25 09:31:32 on dist-test-slave-9pj8"
server_key: "375de12973a193a5b6e40d76e24f2601"
server_key_iv: "3e69ddfdba52e0784a138a50a80e4835"
server_key_version: "encryptionkey@0"
I20260925 09:31:32.092837 30350 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:31:32.096068 30365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:32.097321 30350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:31:32.097486 30350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "be8db5d114de41639654da2cb561c82e"
format_stamp: "Formatted at 2026-09-25 09:31:32 on dist-test-slave-9pj8"
server_key: "375de12973a193a5b6e40d76e24f2601"
server_key_iv: "3e69ddfdba52e0784a138a50a80e4835"
server_key_version: "encryptionkey@0"
I20260925 09:31:32.097676 30350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:32.116055 30350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:32.120208 30350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:32.120486 30350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:32.121393 30350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:31:32.122722 30350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:31:32.122825 30350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:32.122973 30350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:31:32.123044 30350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:32.152211 30350 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:40721
I20260925 09:31:32.152259 30477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:40721 every 8 connection(s)
I20260925 09:31:32.153503 30350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:31:32.156301 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 30350
I20260925 09:31:32.156461 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:31:32.156847 10697 external_mini_cluster.cc:1428] Setting key 1d77cb03598bb98f9cce275cc8650c2b
Sep 25 09:31:32 dist-test-slave-9pj8 krb5kdc[29977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328692, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.67@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:31:32.172273 30213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:32.155925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:52495 (local address 127.10.114.126:46445)
0925 09:31:32.156132 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:32.156150 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:32.157410 (+ 1260us) server_negotiation.cc:407] Connection header received
0925 09:31:32.159535 (+ 2125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:32.159539 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:32.159621 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:32.159743 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:32.161079 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.161820 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.163287 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.163678 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.163841 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:32.166968 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:32.166973 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:32.166978 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:32.167018 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:32.169478 (+ 2460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:32.170130 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:32.170135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:32.170138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:32.170266 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:32.170642 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:32.170647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:32.170650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:32.170930 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:32.171093 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:32.171707 (+ 614us) server_negotiation.cc:299] Negotiation successful
0925 09:31:32.171946 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:31:32.173024 30480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:32.156479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:46445 (local address 127.10.114.67:52495)
0925 09:31:32.157244 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:32.157282 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:32.159285 (+ 2003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:32.159979 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:32.159987 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:32.160445 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:31:32.160929 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:32.160944 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.161953 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.161957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:32.163129 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:32.163140 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.163914 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.163923 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:32.164146 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:32.164782 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:31:32.164805 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:31:32.166789 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:31:32.169607 (+ 2818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:31:32.169615 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:31:32.169627 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:31:32.169999 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:31:32.170367 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:31:32.170373 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:31:32.170376 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:31:32.170533 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:31:32.171057 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:31:32.171065 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:31:32.171360 (+ 295us) client_negotiation.cc:769] Sending connection context
0925 09:31:32.171598 (+ 238us) client_negotiation.cc:240] Negotiation successful
0925 09:31:32.172047 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":303,"threads_started":1}
I20260925 09:31:32.174183 30478 heartbeater.cc:347] Connected to a master server at 127.10.114.126:46445
I20260925 09:31:32.174422 30478 heartbeater.cc:464] Registering TS with master...
I20260925 09:31:32.174999 30478 heartbeater.cc:511] Master 127.10.114.126:46445 requested a full tablet report, sending...
I20260925 09:31:32.176255 30022 ts_manager.cc:194] Registered new tserver with Master: be8db5d114de41639654da2cb561c82e (127.10.114.67:40721)
I20260925 09:31:32.176594 30211 heartbeater.cc:503] Master 127.10.114.126:46445 was elected leader, sending a full tablet report...
I20260925 09:31:32.177156 30022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.114.67@KRBTEST.COM'} at 127.10.114.67:52495
I20260925 09:31:32.183355 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:31:32.199039 30213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:32.187410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33866 (local address 127.10.114.126:46445)
0925 09:31:32.187556 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:32.187560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:32.187947 (+ 387us) server_negotiation.cc:407] Connection header received
0925 09:31:32.188058 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:32.188062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:32.188110 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:32.188182 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:32.189479 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.190111 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.191251 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.191611 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.191686 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:32.193804 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:32.193809 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:32.193812 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:32.193842 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:32.196305 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:32.196893 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:32.196896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:32.196898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:32.197018 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:32.197446 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:32.197452 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:32.197456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:32.197840 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:32.198097 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:32.198479 (+ 382us) server_negotiation.cc:299] Negotiation successful
0925 09:31:32.198693 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 09:31:32.202884 30021 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:31:32.210121 30021 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:31:32.245951 30494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:32.222213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:41647 (local address 127.0.0.1:52968)
0925 09:31:32.231965 (+ 9752us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:32.232014 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:32.232234 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:32.238475 (+ 6241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:32.238482 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:32.238511 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:32.238897 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:32.238915 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.241165 (+ 2250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.241173 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:32.242930 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:32.242949 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.244857 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.244881 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:32.245089 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:32.245170 (+ 81us) client_negotiation.cc:769] Sending connection context
0925 09:31:32.245313 (+ 143us) client_negotiation.cc:240] Negotiation successful
0925 09:31:32.245509 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9397,"thread_start_us":16848,"threads_started":1}
I20260925 09:31:32.246739 30491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:32.222404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52968 (local address 127.10.114.66:41647)
0925 09:31:32.237710 (+ 15306us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:32.237722 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:32.237767 (+ 45us) server_negotiation.cc:407] Connection header received
0925 09:31:32.237902 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:32.237914 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:32.238243 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:32.238471 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:32.239360 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.240970 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.243101 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.244642 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.244769 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:32.244980 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:32.246271 (+ 1291us) server_negotiation.cc:299] Negotiation successful
0925 09:31:32.246451 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15087,"thread_start_us":16254,"threads_started":1}
I20260925 09:31:32.247509 30490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:32.222630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:37387 (local address 127.0.0.1:54340)
0925 09:31:32.239251 (+ 16621us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:32.239268 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:32.239392 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:32.240152 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:32.240157 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:32.240180 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:32.240578 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:32.240589 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.242702 (+ 2113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.242707 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:32.244526 (+ 1819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:32.244539 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.246662 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.246668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:32.246851 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:32.246888 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 09:31:32.246986 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 09:31:32.247121 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16488,"mutex_wait_us":1,"thread_start_us":16648,"threads_started":1}
I20260925 09:31:32.247582 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:32.223270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54340 (local address 127.10.114.65:37387)
0925 09:31:32.239229 (+ 15959us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:32.239241 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:32.239526 (+ 285us) server_negotiation.cc:407] Connection header received
0925 09:31:32.239628 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:32.239635 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:32.239904 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:32.240779 (+ 875us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:32.240979 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.242573 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.244693 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.246489 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.246634 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:32.246861 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:32.247068 (+ 207us) server_negotiation.cc:299] Negotiation successful
0925 09:31:32.247243 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15775,"thread_start_us":15589,"threads_started":1}
I20260925 09:31:32.248646 30492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:32.225145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:40721 (local address 127.0.0.1:59472)
0925 09:31:32.239564 (+ 14419us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:32.239584 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:32.239729 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:32.240657 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:32.240662 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:32.240686 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:32.241137 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:32.241151 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.242960 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.242966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:32.245256 (+ 2290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:32.245269 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.247904 (+ 2635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.247913 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:32.248087 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:32.248115 (+ 28us) client_negotiation.cc:769] Sending connection context
0925 09:31:32.248213 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 09:31:32.248322 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14273,"mutex_wait_us":49,"thread_start_us":6277,"threads_started":1}
I20260925 09:31:32.250350 30280 tablet_service.cc:1467] Processing CreateTablet for tablet 19549d7abcdb4ee8a57948f4f3f57c37 (DEFAULT_TABLE table=test-table [id=10b2f04d60944499b7edf343d8f4e046]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:31:32.251066 30495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:32.225441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59472 (local address 127.10.114.67:40721)
0925 09:31:32.232995 (+ 7554us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:32.233007 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:32.239699 (+ 6692us) server_negotiation.cc:407] Connection header received
0925 09:31:32.239904 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:32.239912 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:32.240162 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:32.240367 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:32.241344 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.242825 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.245429 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.247234 (+ 1805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.247354 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:32.247569 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:32.248279 (+ 710us) server_negotiation.cc:299] Negotiation successful
0925 09:31:32.248448 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7325,"thread_start_us":7136,"threads_started":1}
I20260925 09:31:32.250604 30146 tablet_service.cc:1467] Processing CreateTablet for tablet 19549d7abcdb4ee8a57948f4f3f57c37 (DEFAULT_TABLE table=test-table [id=10b2f04d60944499b7edf343d8f4e046]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:31:32.252151 30280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19549d7abcdb4ee8a57948f4f3f57c37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:32.253979 30413 tablet_service.cc:1467] Processing CreateTablet for tablet 19549d7abcdb4ee8a57948f4f3f57c37 (DEFAULT_TABLE table=test-table [id=10b2f04d60944499b7edf343d8f4e046]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:31:32.252084 30146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19549d7abcdb4ee8a57948f4f3f57c37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:32.255234 30413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19549d7abcdb4ee8a57948f4f3f57c37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:32.262862 30496 tablet_bootstrap.cc:492] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd: Bootstrap starting.
I20260925 09:31:32.265658 30496 tablet_bootstrap.cc:654] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:32.266633 30496 log.cc:824] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:32.270159 30499 tablet_bootstrap.cc:492] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e: Bootstrap starting.
I20260925 09:31:32.271533 30498 tablet_bootstrap.cc:492] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877: Bootstrap starting.
I20260925 09:31:32.273847 30499 tablet_bootstrap.cc:654] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:32.274757 30498 tablet_bootstrap.cc:654] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:32.274922 30496 tablet_bootstrap.cc:492] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd: No bootstrap required, opened a new log
I20260925 09:31:32.275039 30499 log.cc:824] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:32.275177 30496 ts_tablet_manager.cc:1397] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260925 09:31:32.275986 30498 log.cc:824] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:32.279009 30498 tablet_bootstrap.cc:492] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877: No bootstrap required, opened a new log
I20260925 09:31:32.279299 30498 ts_tablet_manager.cc:1397] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260925 09:31:32.282305 30499 tablet_bootstrap.cc:492] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e: No bootstrap required, opened a new log
I20260925 09:31:32.282555 30499 ts_tablet_manager.cc:1397] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260925 09:31:32.285979 30496 raft_consensus.cc:348] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b962fde0d5e143c3af5b9b1ad8bd45cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41647 } } peers { permanent_uuid: "2f5e5d6f1be0422dae9e651a00490877" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 37387 } } peers { permanent_uuid: "be8db5d114de41639654da2cb561c82e" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40721 } }
I20260925 09:31:32.286366 30496 raft_consensus.cc:374] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:32.286471 30496 raft_consensus.cc:729] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b962fde0d5e143c3af5b9b1ad8bd45cd, State: Initialized, Role: FOLLOWER
I20260925 09:31:32.287022 30496 consensus_queue.cc:260] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b962fde0d5e143c3af5b9b1ad8bd45cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41647 } } peers { permanent_uuid: "2f5e5d6f1be0422dae9e651a00490877" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 37387 } } peers { permanent_uuid: "be8db5d114de41639654da2cb561c82e" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40721 } }
I20260925 09:31:32.287788 30498 raft_consensus.cc:348] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b962fde0d5e143c3af5b9b1ad8bd45cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41647 } } peers { permanent_uuid: "2f5e5d6f1be0422dae9e651a00490877" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 37387 } } peers { permanent_uuid: "be8db5d114de41639654da2cb561c82e" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40721 } }
I20260925 09:31:32.288156 30345 heartbeater.cc:503] Master 127.10.114.126:46445 was elected leader, sending a full tablet report...
I20260925 09:31:32.288236 30498 raft_consensus.cc:374] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:32.288343 30498 raft_consensus.cc:729] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f5e5d6f1be0422dae9e651a00490877, State: Initialized, Role: FOLLOWER
I20260925 09:31:32.288708 30496 ts_tablet_manager.cc:1428] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260925 09:31:32.288952 30498 consensus_queue.cc:260] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b962fde0d5e143c3af5b9b1ad8bd45cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41647 } } peers { permanent_uuid: "2f5e5d6f1be0422dae9e651a00490877" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 37387 } } peers { permanent_uuid: "be8db5d114de41639654da2cb561c82e" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40721 } }
I20260925 09:31:32.291807 30498 ts_tablet_manager.cc:1428] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260925 09:31:32.292928 30499 raft_consensus.cc:348] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b962fde0d5e143c3af5b9b1ad8bd45cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41647 } } peers { permanent_uuid: "2f5e5d6f1be0422dae9e651a00490877" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 37387 } } peers { permanent_uuid: "be8db5d114de41639654da2cb561c82e" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40721 } }
I20260925 09:31:32.293340 30499 raft_consensus.cc:374] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:32.293437 30499 raft_consensus.cc:729] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be8db5d114de41639654da2cb561c82e, State: Initialized, Role: FOLLOWER
I20260925 09:31:32.294080 30499 consensus_queue.cc:260] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b962fde0d5e143c3af5b9b1ad8bd45cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41647 } } peers { permanent_uuid: "2f5e5d6f1be0422dae9e651a00490877" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 37387 } } peers { permanent_uuid: "be8db5d114de41639654da2cb561c82e" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40721 } }
I20260925 09:31:32.295094 30478 heartbeater.cc:503] Master 127.10.114.126:46445 was elected leader, sending a full tablet report...
I20260925 09:31:32.295842 30499 ts_tablet_manager.cc:1428] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260925 09:31:32.320422 30503 raft_consensus.cc:482] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:31:32.320694 30503 raft_consensus.cc:504] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b962fde0d5e143c3af5b9b1ad8bd45cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41647 } } peers { permanent_uuid: "2f5e5d6f1be0422dae9e651a00490877" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 37387 } } peers { permanent_uuid: "be8db5d114de41639654da2cb561c82e" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40721 } }
I20260925 09:31:32.322360 30503 leader_election.cc:288] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b962fde0d5e143c3af5b9b1ad8bd45cd (127.10.114.66:41647), be8db5d114de41639654da2cb561c82e (127.10.114.67:40721)
I20260925 09:31:32.329589 30491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:32.322595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:55819 (local address 127.10.114.66:41647)
0925 09:31:32.322828 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:32.322837 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:32.323813 (+ 976us) server_negotiation.cc:407] Connection header received
0925 09:31:32.324069 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:32.324074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:32.324170 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:32.324265 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:32.324962 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.325842 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.327735 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.328551 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.328657 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:32.328701 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:32.329185 (+ 484us) server_negotiation.cc:299] Negotiation successful
0925 09:31:32.329321 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 09:31:32.329798 30505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:32.322854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:40721 (local address 127.10.114.65:37751)
0925 09:31:32.323957 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:32.323976 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:32.324094 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:32.324438 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:32.324444 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:32.324481 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:32.324976 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:32.324994 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.326319 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.326323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:32.327691 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:32.327710 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.328938 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.328942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:32.329083 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:32.329124 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 09:31:32.329210 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 09:31:32.329345 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":988,"mutex_wait_us":18,"thread_start_us":431,"threads_started":1}
I20260925 09:31:32.330195 30495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:32.322855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:37751 (local address 127.10.114.67:40721)
0925 09:31:32.323101 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:32.323110 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:32.324061 (+ 951us) server_negotiation.cc:407] Connection header received
0925 09:31:32.324208 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:32.324216 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:32.324288 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:32.324385 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:32.325129 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.326157 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.327896 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.328737 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.328891 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:32.328966 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:32.329259 (+ 293us) server_negotiation.cc:299] Negotiation successful
0925 09:31:32.329411 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 09:31:32.330343 30506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:32.322534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:41647 (local address 127.10.114.65:55819)
0925 09:31:32.323666 (+ 1132us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:32.323693 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:32.323896 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:32.324292 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:32.324296 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:32.324317 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:32.324795 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:32.324814 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.325985 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.325991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:32.327575 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:32.327588 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.328772 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.328782 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:32.329024 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:32.329054 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 09:31:32.329131 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 09:31:32.329257 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"mutex_wait_us":29,"thread_start_us":1103,"threads_started":1}
I20260925 09:31:32.330987 30433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19549d7abcdb4ee8a57948f4f3f57c37" candidate_uuid: "2f5e5d6f1be0422dae9e651a00490877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be8db5d114de41639654da2cb561c82e" is_pre_election: true
I20260925 09:31:32.331210 30300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19549d7abcdb4ee8a57948f4f3f57c37" candidate_uuid: "2f5e5d6f1be0422dae9e651a00490877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b962fde0d5e143c3af5b9b1ad8bd45cd" is_pre_election: true
I20260925 09:31:32.331422 30433 raft_consensus.cc:2454] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f5e5d6f1be0422dae9e651a00490877 in term 0.
I20260925 09:31:32.331707 30300 raft_consensus.cc:2454] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f5e5d6f1be0422dae9e651a00490877 in term 0.
I20260925 09:31:32.332093 30102 leader_election.cc:302] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f5e5d6f1be0422dae9e651a00490877, be8db5d114de41639654da2cb561c82e; no voters:
I20260925 09:31:32.332432 30503 raft_consensus.cc:2781] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:31:32.332568 30503 raft_consensus.cc:482] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:31:32.332671 30503 raft_consensus.cc:3037] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:32.334637 30503 raft_consensus.cc:504] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b962fde0d5e143c3af5b9b1ad8bd45cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41647 } } peers { permanent_uuid: "2f5e5d6f1be0422dae9e651a00490877" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 37387 } } peers { permanent_uuid: "be8db5d114de41639654da2cb561c82e" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40721 } }
I20260925 09:31:32.335156 30503 leader_election.cc:288] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [CANDIDATE]: Term 1 election: Requested vote from peers b962fde0d5e143c3af5b9b1ad8bd45cd (127.10.114.66:41647), be8db5d114de41639654da2cb561c82e (127.10.114.67:40721)
I20260925 09:31:32.335559 30300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19549d7abcdb4ee8a57948f4f3f57c37" candidate_uuid: "2f5e5d6f1be0422dae9e651a00490877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b962fde0d5e143c3af5b9b1ad8bd45cd"
I20260925 09:31:32.335565 30433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19549d7abcdb4ee8a57948f4f3f57c37" candidate_uuid: "2f5e5d6f1be0422dae9e651a00490877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be8db5d114de41639654da2cb561c82e"
I20260925 09:31:32.335747 30300 raft_consensus.cc:3037] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:32.335752 30433 raft_consensus.cc:3037] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:32.337817 30433 raft_consensus.cc:2454] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f5e5d6f1be0422dae9e651a00490877 in term 1.
I20260925 09:31:32.338040 30300 raft_consensus.cc:2454] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f5e5d6f1be0422dae9e651a00490877 in term 1.
I20260925 09:31:32.338138 30102 leader_election.cc:302] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f5e5d6f1be0422dae9e651a00490877, be8db5d114de41639654da2cb561c82e; no voters:
I20260925 09:31:32.338392 30503 raft_consensus.cc:2781] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:32.338799 30503 raft_consensus.cc:686] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [term 1 LEADER]: Becoming Leader. State: Replica: 2f5e5d6f1be0422dae9e651a00490877, State: Running, Role: LEADER
I20260925 09:31:32.339222 30503 consensus_queue.cc:237] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b962fde0d5e143c3af5b9b1ad8bd45cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41647 } } peers { permanent_uuid: "2f5e5d6f1be0422dae9e651a00490877" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 37387 } } peers { permanent_uuid: "be8db5d114de41639654da2cb561c82e" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40721 } }
I20260925 09:31:32.343809 30020 catalog_manager.cc:5495] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f5e5d6f1be0422dae9e651a00490877 (127.10.114.65). New cstate: current_term: 1 leader_uuid: "2f5e5d6f1be0422dae9e651a00490877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b962fde0d5e143c3af5b9b1ad8bd45cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f5e5d6f1be0422dae9e651a00490877" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 37387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be8db5d114de41639654da2cb561c82e" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260925 09:31:32.360531 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:32.355614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54342 (local address 127.10.114.65:37387)
0925 09:31:32.355851 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:32.355856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:32.355874 (+ 18us) server_negotiation.cc:407] Connection header received
0925 09:31:32.356045 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:32.356049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:32.356120 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:32.356294 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:31:32.356609 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.357329 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.358609 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.358994 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.359092 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:32.359541 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:31:32.360079 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:31:32.360204 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:32.360290 (+ 86us) server_negotiation.cc:299] Negotiation successful
0925 09:31:32.360350 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 09:31:32.367590 30300 raft_consensus.cc:1261] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd [term 1 FOLLOWER]: Refusing update from remote peer 2f5e5d6f1be0422dae9e651a00490877: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:31:32.367594 30433 raft_consensus.cc:1261] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e [term 1 FOLLOWER]: Refusing update from remote peer 2f5e5d6f1be0422dae9e651a00490877: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:31:32.368558 30503 consensus_queue.cc:1035] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be8db5d114de41639654da2cb561c82e" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:31:32.368829 30507 consensus_queue.cc:1035] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b962fde0d5e143c3af5b9b1ad8bd45cd" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 41647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:31:32.380592 30511 mvcc.cc:205] Tried to move back new op lower bound from 7333186323929313280 to 7333186323825225728. Current Snapshot: MvccSnapshot[applied={T|T < 7333186323929313280}]
I20260925 09:31:32.380697 30512 mvcc.cc:205] Tried to move back new op lower bound from 7333186323929313280 to 7333186323825225728. Current Snapshot: MvccSnapshot[applied={T|T < 7333186323929313280}]
I20260925 09:31:32.392078 30022 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:31:32.392298 30022 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:31:32.395655 30022 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae4c1151b72247ce809950c30880748c: Sending DeleteTablet for 3 replicas of tablet 19549d7abcdb4ee8a57948f4f3f57c37
I20260925 09:31:32.396871 30413 tablet_service.cc:1514] Processing DeleteTablet for tablet 19549d7abcdb4ee8a57948f4f3f57c37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:31:32 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:59472
I20260925 09:31:32.396858 30280 tablet_service.cc:1514] Processing DeleteTablet for tablet 19549d7abcdb4ee8a57948f4f3f57c37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:31:32 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:52968
I20260925 09:31:32.396879 30146 tablet_service.cc:1514] Processing DeleteTablet for tablet 19549d7abcdb4ee8a57948f4f3f57c37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:31:32 UTC) from {username='kudu', principal='kudu/127.10.114.126@KRBTEST.COM'} at 127.0.0.1:54340
I20260925 09:31:32.400247 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 30083
I20260925 09:31:32.417198 30522 tablet_replica.cc:330] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877: stopping tablet replica
I20260925 09:31:32.417793 30522 raft_consensus.cc:2229] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:31:32.418759 30522 raft_consensus.cc:2258] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:31:32.421043 30521 tablet_replica.cc:330] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd: stopping tablet replica
I20260925 09:31:32.421464 30521 raft_consensus.cc:2229] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:31:32.421973 30521 raft_consensus.cc:2258] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:31:32.422222 30522 ts_tablet_manager.cc:1905] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:31:32.422335 30520 tablet_replica.cc:330] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e: stopping tablet replica
I20260925 09:31:32.422744 30520 raft_consensus.cc:2229] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:31:32.423223 30520 raft_consensus.cc:2258] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:31:32.425065 30521 ts_tablet_manager.cc:1905] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 25 09:31:32 dist-test-slave-9pj8 krb5kdc[29977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.65@KRBTEST.COM
I20260925 09:31:32.429378 30520 ts_tablet_manager.cc:1905] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:31:32.431411 30522 ts_tablet_manager.cc:1918] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:31:32.431572 30522 log.cc:1196] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/19549d7abcdb4ee8a57948f4f3f57c37
I20260925 09:31:32.431874 30521 ts_tablet_manager.cc:1918] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:31:32.432049 30521 log.cc:1196] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/19549d7abcdb4ee8a57948f4f3f57c37
I20260925 09:31:32.432207 30522 ts_tablet_manager.cc:1939] T 19549d7abcdb4ee8a57948f4f3f57c37 P 2f5e5d6f1be0422dae9e651a00490877: Deleting consensus metadata
I20260925 09:31:32.432574 30521 ts_tablet_manager.cc:1939] T 19549d7abcdb4ee8a57948f4f3f57c37 P b962fde0d5e143c3af5b9b1ad8bd45cd: Deleting consensus metadata
I20260925 09:31:32.434103 30008 catalog_manager.cc:4841] TS 2f5e5d6f1be0422dae9e651a00490877 (127.10.114.65:37387): tablet 19549d7abcdb4ee8a57948f4f3f57c37 (table test-table [id=10b2f04d60944499b7edf343d8f4e046]) successfully deleted
I20260925 09:31:32.434823 30007 catalog_manager.cc:4841] TS b962fde0d5e143c3af5b9b1ad8bd45cd (127.10.114.66:41647): tablet 19549d7abcdb4ee8a57948f4f3f57c37 (table test-table [id=10b2f04d60944499b7edf343d8f4e046]) successfully deleted
I20260925 09:31:32.436007 30520 ts_tablet_manager.cc:1918] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:31:32.436157 30520 log.cc:1196] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/19549d7abcdb4ee8a57948f4f3f57c37
I20260925 09:31:32.436640 30520 ts_tablet_manager.cc:1939] T 19549d7abcdb4ee8a57948f4f3f57c37 P be8db5d114de41639654da2cb561c82e: Deleting consensus metadata
I20260925 09:31:32.437040 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:32.402238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54356 (local address 127.10.114.65:37387)
0925 09:31:32.407374 (+ 5136us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:32.407384 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:32.407416 (+ 32us) server_negotiation.cc:407] Connection header received
0925 09:31:32.407647 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:32.407656 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:32.407764 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:32.407870 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:32.410111 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.411663 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.421205 (+ 9542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.421908 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.422007 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:32.427364 (+ 5357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:32.427377 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:32.427392 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:32.427442 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:32.431444 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:32.434465 (+ 3021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:32.434473 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:32.434477 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:32.434636 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:32.435139 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:32.435144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:32.435146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:32.435517 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:32.435874 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:32.436347 (+ 473us) server_negotiation.cc:299] Negotiation successful
0925 09:31:32.436585 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4964}
I20260925 09:31:32.437685 30206 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:32.438187 30009 catalog_manager.cc:4841] TS be8db5d114de41639654da2cb561c82e (127.10.114.67:40721): tablet 19549d7abcdb4ee8a57948f4f3f57c37 (table test-table [id=10b2f04d60944499b7edf343d8f4e046]) successfully deleted
I20260925 09:31:32.534370 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 30083
I20260925 09:31:32.556366 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 30217
Sep 25 09:31:32 dist-test-slave-9pj8 krb5kdc[29977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.66@KRBTEST.COM
I20260925 09:31:32.571233 30491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:32.557132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52970 (local address 127.10.114.66:41647)
0925 09:31:32.557381 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:32.557386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:32.557405 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:31:32.557598 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:32.557606 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:32.557706 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:32.557838 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:32.559412 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.560309 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.561607 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.561977 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.562076 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:32.566041 (+ 3965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:32.566052 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:32.566066 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:32.566100 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:32.568878 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:32.569486 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:32.569493 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:32.569495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:32.569614 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:32.569941 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:32.569944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:32.569946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:32.570293 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:32.570489 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:32.570762 (+ 273us) server_negotiation.cc:299] Negotiation successful
0925 09:31:32.570905 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260925 09:31:32.571871 30340 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:32.656913 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 30217
I20260925 09:31:32.674024 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 30350
Sep 25 09:31:32 dist-test-slave-9pj8 krb5kdc[29977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.67@KRBTEST.COM
I20260925 09:31:32.688537 30495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:32.674724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59482 (local address 127.10.114.67:40721)
0925 09:31:32.674903 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:32.674908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:32.674925 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:31:32.675016 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:32.675020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:32.675092 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:32.675173 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:32.676493 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.677201 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.678425 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:32.678795 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:32.678927 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:32.683236 (+ 4309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:32.683258 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:32.683280 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:32.683333 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:32.685870 (+ 2537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:32.686588 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:32.686597 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:32.686600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:32.686745 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:32.687105 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:32.687110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:32.687113 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:32.687449 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:32.687749 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:32.688042 (+ 293us) server_negotiation.cc:299] Negotiation successful
0925 09:31:32.688250 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 09:31:32.689217 30473 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:32.776332 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 30350
I20260925 09:31:32.794255 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 29990
I20260925 09:31:32.795372 30052 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:32.904785 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 29990
2026-09-25T09:31:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4102 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-25T09:31:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:31:32Z Disabled control of system clock
I20260925 09:31:32.977381 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:39781
--webserver_interface=127.10.114.126
--webserver_port=0
--builtin_ntp_servers=127.10.114.84:40259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:39781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 09:31:33.311093 30531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:33.311385 30531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:33.311488 30531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:33.319018 30531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:31:33.319116 30531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:33.319152 30531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:33.319199 30531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:31:33.319265 30531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:31:33.328074 30531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:40259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:39781
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:39781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:33.329373 30531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:33.331073 30531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:33.342336 30536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:33.342428 30539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:33.342450 30537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:33.343029 30531 server_base.cc:1034] running on GCE node
I20260925 09:31:33.343760 30531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:33.345130 30531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:33.346351 30531 hybrid_clock.cc:648] HybridClock initialized: now 1790328693346292 us; error 70 us; skew 500 ppm
I20260925 09:31:33.346679 30531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:31:33.348748 30531 webserver.cc:460] Webserver started at http://127.10.114.126:39741/ using document root <none> and password file <none>
I20260925 09:31:33.349375 30531 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:33.349498 30531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:33.349795 30531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:33.352550 30531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "d878527386fa43398a7c1909e7ddaaa5"
format_stamp: "Formatted at 2026-09-25 09:31:33 on dist-test-slave-9pj8"
server_key: "53517587d4169690c5932af9933cddfa"
server_key_iv: "be33c154655df424b3dbe0dff4b4699f"
server_key_version: "encryptionkey@0"
I20260925 09:31:33.353303 30531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "d878527386fa43398a7c1909e7ddaaa5"
format_stamp: "Formatted at 2026-09-25 09:31:33 on dist-test-slave-9pj8"
server_key: "53517587d4169690c5932af9933cddfa"
server_key_iv: "be33c154655df424b3dbe0dff4b4699f"
server_key_version: "encryptionkey@0"
I20260925 09:31:33.358091 30531 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260925 09:31:33.361285 30545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:33.362535 30531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 09:31:33.362730 30531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "d878527386fa43398a7c1909e7ddaaa5"
format_stamp: "Formatted at 2026-09-25 09:31:33 on dist-test-slave-9pj8"
server_key: "53517587d4169690c5932af9933cddfa"
server_key_iv: "be33c154655df424b3dbe0dff4b4699f"
server_key_version: "encryptionkey@0"
I20260925 09:31:33.362941 30531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:33.383379 30531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:33.384163 30531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:33.384445 30531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:33.401998 30531 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:39781
I20260925 09:31:33.402016 30596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:39781 every 8 connection(s)
I20260925 09:31:33.403261 30531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:31:33.406881 30597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:33.407949 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 30531
I20260925 09:31:33.408169 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:31:33.408793 10697 external_mini_cluster.cc:1428] Setting key 797b5fadfe3cbcbaefb900d3b916f7d0
I20260925 09:31:33.420920 30597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5: Bootstrap starting.
I20260925 09:31:33.422132 30600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:33.410801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35642 (local address 127.10.114.126:39781)
0925 09:31:33.411846 (+ 1045us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:33.411861 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:33.411904 (+ 43us) server_negotiation.cc:407] Connection header received
0925 09:31:33.413509 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:33.413529 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:33.413960 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:33.414351 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:33.414930 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:33.416163 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:33.417980 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:33.418656 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:33.418799 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:33.420024 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:33.420031 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:33.420043 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:33.420555 (+ 512us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:33.420649 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:33.420670 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:33.420774 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:33.420994 (+ 220us) server_negotiation.cc:299] Negotiation successful
0925 09:31:33.421282 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":366,"threads_started":1}
I20260925 09:31:33.424374 30597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:33.425558 30597 log.cc:824] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:33.428120 30597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5: No bootstrap required, opened a new log
I20260925 09:31:33.435161 30597 raft_consensus.cc:348] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d878527386fa43398a7c1909e7ddaaa5" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39781 } }
I20260925 09:31:33.435456 30597 raft_consensus.cc:374] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:33.435571 30597 raft_consensus.cc:729] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d878527386fa43398a7c1909e7ddaaa5, State: Initialized, Role: FOLLOWER
I20260925 09:31:33.436080 30597 consensus_queue.cc:260] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d878527386fa43398a7c1909e7ddaaa5" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39781 } }
I20260925 09:31:33.436305 30597 raft_consensus.cc:388] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:31:33.436393 30597 raft_consensus.cc:482] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:31:33.436496 30597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:33.438879 30597 raft_consensus.cc:504] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d878527386fa43398a7c1909e7ddaaa5" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39781 } }
I20260925 09:31:33.439309 30597 leader_election.cc:302] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d878527386fa43398a7c1909e7ddaaa5; no voters:
I20260925 09:31:33.439900 30597 leader_election.cc:288] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:31:33.440048 30602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:33.440750 30602 raft_consensus.cc:686] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5 [term 1 LEADER]: Becoming Leader. State: Replica: d878527386fa43398a7c1909e7ddaaa5, State: Running, Role: LEADER
I20260925 09:31:33.441385 30602 consensus_queue.cc:237] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d878527386fa43398a7c1909e7ddaaa5" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39781 } }
I20260925 09:31:33.442059 30597 sys_catalog.cc:564] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:31:33.444188 30604 sys_catalog.cc:455] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d878527386fa43398a7c1909e7ddaaa5. Latest consensus state: current_term: 1 leader_uuid: "d878527386fa43398a7c1909e7ddaaa5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d878527386fa43398a7c1909e7ddaaa5" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39781 } } }
I20260925 09:31:33.444649 30604 sys_catalog.cc:458] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:33.444902 30603 sys_catalog.cc:455] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d878527386fa43398a7c1909e7ddaaa5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d878527386fa43398a7c1909e7ddaaa5" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39781 } } }
I20260925 09:31:33.445323 30603 sys_catalog.cc:458] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:33.446843 30611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:31:33.451608 30611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:31:33.460031 30611 catalog_manager.cc:1347] Generated new cluster ID: 3c5c7833e5d54ddb91ef8eb90edeada8
I20260925 09:31:33.460134 30611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:31:33.481050 30611 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:31:33.482462 30611 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:31:33.488879 30611 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5: Generated new TSK 0
I20260925 09:31:33.489763 30611 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:31:33.505200 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.65:0
--local_ip_for_outbound_sockets=127.10.114.65
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39781
--builtin_ntp_servers=127.10.114.84:40259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 09:31:33.834259 30621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:33.834504 30621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:33.834642 30621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:33.841544 30621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:33.841636 30621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:33.841804 30621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.65
I20260925 09:31:33.850422 30621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:40259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.114.65
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39781
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.65
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:33.851596 30621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:33.853385 30621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:33.866633 30626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:33.866689 30629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:33.866634 30627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:33.867841 30621 server_base.cc:1034] running on GCE node
I20260925 09:31:33.868484 30621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:33.869445 30621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:33.870694 30621 hybrid_clock.cc:648] HybridClock initialized: now 1790328693870625 us; error 31 us; skew 500 ppm
I20260925 09:31:33.871057 30621 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:31:33.879242 30621 webserver.cc:460] Webserver started at http://127.10.114.65:44895/ using document root <none> and password file <none>
I20260925 09:31:33.879997 30621 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:33.880108 30621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:33.880429 30621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:33.883186 30621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-0/data/instance:
uuid: "499978cf17764a0b9f4e1afbeb13ff1b"
format_stamp: "Formatted at 2026-09-25 09:31:33 on dist-test-slave-9pj8"
server_key: "ea9ee3e3fee776a549c099c44970f16b"
server_key_iv: "174867f2e741e7302ed5205f92965566"
server_key_version: "encryptionkey@0"
I20260925 09:31:33.884022 30621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance:
uuid: "499978cf17764a0b9f4e1afbeb13ff1b"
format_stamp: "Formatted at 2026-09-25 09:31:33 on dist-test-slave-9pj8"
server_key: "ea9ee3e3fee776a549c099c44970f16b"
server_key_iv: "174867f2e741e7302ed5205f92965566"
server_key_version: "encryptionkey@0"
I20260925 09:31:33.888792 30621 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 09:31:33.892282 30635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:33.893678 30621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260925 09:31:33.893918 30621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-0/wal
uuid: "499978cf17764a0b9f4e1afbeb13ff1b"
format_stamp: "Formatted at 2026-09-25 09:31:33 on dist-test-slave-9pj8"
server_key: "ea9ee3e3fee776a549c099c44970f16b"
server_key_iv: "174867f2e741e7302ed5205f92965566"
server_key_version: "encryptionkey@0"
I20260925 09:31:33.894187 30621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:33.926980 30621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:33.927836 30621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:33.928122 30621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:33.929054 30621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:31:33.930413 30621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:31:33.930539 30621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:33.930656 30621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:31:33.930744 30621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:33.962625 30621 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.65:34807
I20260925 09:31:33.962728 30747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.65:34807 every 8 connection(s)
I20260925 09:31:33.964035 30621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-0/data/info.pb
I20260925 09:31:33.970916 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 30621
I20260925 09:31:33.971086 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-0/wal/instance
I20260925 09:31:33.971505 10697 external_mini_cluster.cc:1428] Setting key c0b4c9c9d4cd5c8f63eab3ee635adb41
I20260925 09:31:33.973996 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.66:0
--local_ip_for_outbound_sockets=127.10.114.66
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39781
--builtin_ntp_servers=127.10.114.84:40259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 09:31:33.976701 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:33.966446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:40131 (local address 127.10.114.126:39781)
0925 09:31:33.966856 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:33.966860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:33.968074 (+ 1214us) server_negotiation.cc:407] Connection header received
0925 09:31:33.969874 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:33.969880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:33.969965 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:33.970079 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:33.971166 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:33.972008 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:33.973764 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:33.974213 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:33.974340 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:33.975928 (+ 1588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:33.975934 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:33.975937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:33.975955 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:33.975996 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:33.976028 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:33.976112 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:33.976366 (+ 254us) server_negotiation.cc:299] Negotiation successful
0925 09:31:33.976443 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":220,"threads_started":1}
I20260925 09:31:33.977964 30751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:33.966981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39781 (local address 127.10.114.65:40131)
0925 09:31:33.967925 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:33.967968 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:33.969597 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:33.970309 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:33.970317 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:33.970399 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:31:33.970961 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:33.971003 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:33.972162 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:33.972168 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:33.973607 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:33.973623 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:33.974443 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:33.974452 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:33.974785 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:33.975561 (+ 776us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:31:33.975567 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:31:33.975619 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:31:33.975725 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:31:33.976135 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:31:33.976148 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:31:33.976158 (+ 10us) client_negotiation.cc:769] Sending connection context
0925 09:31:33.976340 (+ 182us) client_negotiation.cc:240] Negotiation successful
0925 09:31:33.976735 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":278,"threads_started":1}
I20260925 09:31:33.979422 30748 heartbeater.cc:347] Connected to a master server at 127.10.114.126:39781
I20260925 09:31:33.979722 30748 heartbeater.cc:464] Registering TS with master...
I20260925 09:31:33.980508 30748 heartbeater.cc:511] Master 127.10.114.126:39781 requested a full tablet report, sending...
I20260925 09:31:33.982564 30562 ts_manager.cc:194] Registered new tserver with Master: 499978cf17764a0b9f4e1afbeb13ff1b (127.10.114.65:34807)
I20260925 09:31:33.984086 30562 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.114.65:40131
W20260925 09:31:34.294915 30752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:34.295176 30752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:34.295346 30752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:34.302345 30752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:34.302431 30752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:34.302623 30752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.66
I20260925 09:31:34.311472 30752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:40259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.114.66
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39781
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.66
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:34.312666 30752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:34.314435 30752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:34.327384 30760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:34.327425 30758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:34.327519 30757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:34.328166 30752 server_base.cc:1034] running on GCE node
I20260925 09:31:34.328759 30752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:34.329653 30752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:34.330894 30752 hybrid_clock.cc:648] HybridClock initialized: now 1790328694330841 us; error 45 us; skew 500 ppm
I20260925 09:31:34.331209 30752 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:31:34.333389 30752 webserver.cc:460] Webserver started at http://127.10.114.66:33221/ using document root <none> and password file <none>
I20260925 09:31:34.334079 30752 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:34.334216 30752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:34.334547 30752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:34.337424 30752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-1/data/instance:
uuid: "e3fb93bfcedf425e8aa87768a8190b33"
format_stamp: "Formatted at 2026-09-25 09:31:34 on dist-test-slave-9pj8"
server_key: "950b29fa58990e390f655de18b261c32"
server_key_iv: "fe19d1486d3d344bf308bf41791e7491"
server_key_version: "encryptionkey@0"
I20260925 09:31:34.338217 30752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance:
uuid: "e3fb93bfcedf425e8aa87768a8190b33"
format_stamp: "Formatted at 2026-09-25 09:31:34 on dist-test-slave-9pj8"
server_key: "950b29fa58990e390f655de18b261c32"
server_key_iv: "fe19d1486d3d344bf308bf41791e7491"
server_key_version: "encryptionkey@0"
I20260925 09:31:34.343430 30752 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 09:31:34.346616 30766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:34.347965 30752 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 09:31:34.348101 30752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-1/wal
uuid: "e3fb93bfcedf425e8aa87768a8190b33"
format_stamp: "Formatted at 2026-09-25 09:31:34 on dist-test-slave-9pj8"
server_key: "950b29fa58990e390f655de18b261c32"
server_key_iv: "fe19d1486d3d344bf308bf41791e7491"
server_key_version: "encryptionkey@0"
I20260925 09:31:34.348255 30752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:34.365036 30752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:34.365645 30752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:34.365950 30752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:34.366869 30752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:31:34.368227 30752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:31:34.368359 30752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:34.368470 30752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:31:34.368549 30752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:34.401082 30752 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.66:35141
I20260925 09:31:34.401158 30878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.66:35141 every 8 connection(s)
I20260925 09:31:34.402567 30752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-1/data/info.pb
I20260925 09:31:34.408401 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 30752
I20260925 09:31:34.408557 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-1/wal/instance
I20260925 09:31:34.409061 10697 external_mini_cluster.cc:1428] Setting key bf2103d072b32413254f77cba10c3618
I20260925 09:31:34.412048 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.114.67:0
--local_ip_for_outbound_sockets=127.10.114.67
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39781
--builtin_ntp_servers=127.10.114.84:40259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 09:31:34.417546 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:34.405253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.66:50277 (local address 127.10.114.126:39781)
0925 09:31:34.405434 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:34.405439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:34.407164 (+ 1725us) server_negotiation.cc:407] Connection header received
0925 09:31:34.409064 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:34.409070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:34.409159 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:34.409297 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:34.410557 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.411868 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.414351 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.414973 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.415084 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:34.416691 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:34.416696 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:34.416701 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:34.416719 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:34.416753 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:34.416767 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:34.416847 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:34.417106 (+ 259us) server_negotiation.cc:299] Negotiation successful
0925 09:31:34.417195 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 09:31:34.418949 30881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:34.405769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39781 (local address 127.10.114.66:50277)
0925 09:31:34.407000 (+ 1231us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:34.407044 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:34.408796 (+ 1752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:34.409519 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:34.409527 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:34.409654 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:31:34.410385 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:34.410408 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.412025 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.412033 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:34.414197 (+ 2164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:34.414215 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.415193 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.415201 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:34.415509 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:34.416363 (+ 854us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:31:34.416368 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:31:34.416415 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:31:34.416512 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:31:34.416867 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:31:34.416878 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:31:34.416887 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 09:31:34.417069 (+ 182us) client_negotiation.cc:240] Negotiation successful
0925 09:31:34.417509 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":286,"threads_started":1}
I20260925 09:31:34.420281 30879 heartbeater.cc:347] Connected to a master server at 127.10.114.126:39781
I20260925 09:31:34.420557 30879 heartbeater.cc:464] Registering TS with master...
I20260925 09:31:34.421375 30879 heartbeater.cc:511] Master 127.10.114.126:39781 requested a full tablet report, sending...
I20260925 09:31:34.422811 30562 ts_manager.cc:194] Registered new tserver with Master: e3fb93bfcedf425e8aa87768a8190b33 (127.10.114.66:35141)
I20260925 09:31:34.423600 30562 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.114.66:50277
W20260925 09:31:34.738415 30882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:34.738713 30882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:34.738873 30882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:34.745997 30882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:34.746117 30882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:34.746268 30882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.114.67
I20260925 09:31:34.754925 30882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:40259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.114.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.114.67
--webserver_port=0
--tserver_master_addrs=127.10.114.126:39781
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.114.67
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:34.756167 30882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:34.757843 30882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:34.770674 30887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:34.770720 30888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:34.771008 30890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:34.771317 30882 server_base.cc:1034] running on GCE node
I20260925 09:31:34.772015 30882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:34.772957 30882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:34.774169 30882 hybrid_clock.cc:648] HybridClock initialized: now 1790328694774132 us; error 34 us; skew 500 ppm
I20260925 09:31:34.774581 30882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:31:34.777338 30882 webserver.cc:460] Webserver started at http://127.10.114.67:39915/ using document root <none> and password file <none>
I20260925 09:31:34.778046 30882 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:34.778167 30882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:34.778498 30882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:34.781415 30882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-2/data/instance:
uuid: "37b530c91bd540f6bbc5c660f1392785"
format_stamp: "Formatted at 2026-09-25 09:31:34 on dist-test-slave-9pj8"
server_key: "251ac9fb10bff896e5de953b77f3269c"
server_key_iv: "9656e7b33bce55ecd8fa1f47ba9f865e"
server_key_version: "encryptionkey@0"
I20260925 09:31:34.782209 30882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance:
uuid: "37b530c91bd540f6bbc5c660f1392785"
format_stamp: "Formatted at 2026-09-25 09:31:34 on dist-test-slave-9pj8"
server_key: "251ac9fb10bff896e5de953b77f3269c"
server_key_iv: "9656e7b33bce55ecd8fa1f47ba9f865e"
server_key_version: "encryptionkey@0"
I20260925 09:31:34.787073 30882 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:31:34.790702 30896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:34.792001 30882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:31:34.792165 30882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-2/wal
uuid: "37b530c91bd540f6bbc5c660f1392785"
format_stamp: "Formatted at 2026-09-25 09:31:34 on dist-test-slave-9pj8"
server_key: "251ac9fb10bff896e5de953b77f3269c"
server_key_iv: "9656e7b33bce55ecd8fa1f47ba9f865e"
server_key_version: "encryptionkey@0"
I20260925 09:31:34.792367 30882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:34.813244 30882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:34.813985 30882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:34.814283 30882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:34.815157 30882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:31:34.816552 30882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:31:34.816679 30882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:34.816813 30882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:31:34.816955 30882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260925 09:31:34.849601 30882 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.67:37705
I20260925 09:31:34.849655 31008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.67:37705 every 8 connection(s)
I20260925 09:31:34.850907 30882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-2/data/info.pb
I20260925 09:31:34.857467 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 30882
I20260925 09:31:34.857653 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-2/wal/instance
I20260925 09:31:34.858158 10697 external_mini_cluster.cc:1428] Setting key 0f30e3d13a95d2bccff4bf115dd90cb6
I20260925 09:31:34.864282 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:34.853743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.67:40193 (local address 127.10.114.126:39781)
0925 09:31:34.853926 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:34.853930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:34.855293 (+ 1363us) server_negotiation.cc:407] Connection header received
0925 09:31:34.857107 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:34.857111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:34.857178 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:34.857283 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:34.858737 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.859701 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.861526 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.861935 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.862037 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:34.863281 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:34.863286 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:34.863289 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:34.863306 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:34.863337 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:34.863348 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:34.863420 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:34.863694 (+ 274us) server_negotiation.cc:299] Negotiation successful
0925 09:31:34.864020 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 09:31:34.864897 31011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:34.854233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39781 (local address 127.10.114.67:40193)
0925 09:31:34.855126 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:34.855159 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:34.856836 (+ 1677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:34.857555 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:34.857568 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:34.857705 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:31:34.858503 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:34.858526 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.859920 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.859924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:34.861402 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:34.861412 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.862137 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.862142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:34.862360 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:34.862987 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:31:34.862989 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:31:34.863029 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:31:34.863105 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:31:34.863431 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:31:34.863438 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:31:34.863443 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 09:31:34.863572 (+ 129us) client_negotiation.cc:240] Negotiation successful
0925 09:31:34.863909 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":255,"threads_started":1}
I20260925 09:31:34.866128 31009 heartbeater.cc:347] Connected to a master server at 127.10.114.126:39781
I20260925 09:31:34.866429 31009 heartbeater.cc:464] Registering TS with master...
I20260925 09:31:34.867121 31009 heartbeater.cc:511] Master 127.10.114.126:39781 requested a full tablet report, sending...
I20260925 09:31:34.868515 30562 ts_manager.cc:194] Registered new tserver with Master: 37b530c91bd540f6bbc5c660f1392785 (127.10.114.67:37705)
I20260925 09:31:34.869268 30562 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.114.67:40193
I20260925 09:31:34.873390 10697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:31:34.883204 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:34.877669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44196 (local address 127.10.114.126:39781)
0925 09:31:34.877930 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:34.877934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:34.878353 (+ 419us) server_negotiation.cc:407] Connection header received
0925 09:31:34.878491 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:34.878494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:34.878561 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:34.878664 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:34.879075 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.879735 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.881133 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.881530 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.881635 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:34.882626 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:34.882630 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:34.882633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:34.882647 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:34.882693 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:34.882704 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:34.882772 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:34.882934 (+ 162us) server_negotiation.cc:299] Negotiation successful
0925 09:31:34.882996 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260925 09:31:34.886904 30562 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:31:34.893639 30562 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:31:34.923102 31022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:34.905972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43742 (local address 127.10.114.65:34807)
0925 09:31:34.913041 (+ 7069us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:34.913049 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:34.913075 (+ 26us) server_negotiation.cc:407] Connection header received
0925 09:31:34.913165 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:34.913171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:34.913419 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:34.914382 (+ 963us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:34.914641 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.916411 (+ 1770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.920236 (+ 3825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.921741 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.921886 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:34.922130 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:34.922568 (+ 438us) server_negotiation.cc:299] Negotiation successful
0925 09:31:34.922732 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6919,"thread_start_us":6813,"threads_started":1}
I20260925 09:31:34.923105 31021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:34.905984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:34807 (local address 127.0.0.1:43742)
0925 09:31:34.912032 (+ 6048us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:34.912057 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:34.912202 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:34.913656 (+ 1454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:34.913667 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:34.913702 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:34.914178 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:34.914189 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.916576 (+ 2387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.916582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:34.919688 (+ 3106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:34.919701 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.921943 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.921950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:34.922173 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:34.922255 (+ 82us) client_negotiation.cc:769] Sending connection context
0925 09:31:34.922369 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 09:31:34.922650 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5891,"mutex_wait_us":218,"spinlock_wait_cycles":1792,"thread_start_us":747,"threads_started":1}
I20260925 09:31:34.923130 31023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:34.905926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:35141 (local address 127.0.0.1:48124)
0925 09:31:34.907217 (+ 1291us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:34.907276 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:34.907659 (+ 383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:34.913419 (+ 5760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:34.913427 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:34.913478 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:34.914095 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:34.914116 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.917505 (+ 3389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.917513 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:34.920500 (+ 2987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:34.920513 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.922069 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.922076 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:34.922258 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:34.922284 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 09:31:34.922369 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 09:31:34.922650 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":947,"mutex_wait_us":159,"spinlock_wait_cycles":6784,"thread_start_us":5747,"threads_started":1}
I20260925 09:31:34.924299 31024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:34.906572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48124 (local address 127.10.114.66:35141)
0925 09:31:34.912745 (+ 6173us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:34.912755 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:34.912801 (+ 46us) server_negotiation.cc:407] Connection header received
0925 09:31:34.912904 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:34.912912 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:34.913181 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:34.915362 (+ 2181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:34.915614 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.917336 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.920664 (+ 3328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.921871 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.923519 (+ 1648us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:34.923685 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:34.923731 (+ 46us) server_negotiation.cc:299] Negotiation successful
0925 09:31:34.923930 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5991,"thread_start_us":5873,"threads_started":1}
I20260925 09:31:34.927453 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:34.913136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:37705 (local address 127.0.0.1:51502)
0925 09:31:34.919982 (+ 6846us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:34.919997 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:34.920137 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:34.920800 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:34.920805 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:34.920831 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:34.921155 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:34.921165 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.923118 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.923126 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:34.925233 (+ 2107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:34.925249 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.926592 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.926599 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:34.926791 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:34.926832 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 09:31:34.926960 (+ 128us) client_negotiation.cc:240] Negotiation successful
0925 09:31:34.927085 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6436,"thread_start_us":6868,"threads_started":1}
I20260925 09:31:34.928151 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:34.912643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51502 (local address 127.10.114.67:37705)
0925 09:31:34.919288 (+ 6645us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:34.919294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:34.920267 (+ 973us) server_negotiation.cc:407] Connection header received
0925 09:31:34.920381 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:34.920388 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:34.920611 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:34.920820 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:34.921352 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.922928 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.925458 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:34.926379 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:34.926561 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:34.926874 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:34.927599 (+ 725us) server_negotiation.cc:299] Negotiation successful
0925 09:31:34.927805 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6488,"thread_start_us":6339,"threads_started":1}
I20260925 09:31:34.927793 30683 tablet_service.cc:1467] Processing CreateTablet for tablet 15d71270c9134dbbad7b931135edb7df (DEFAULT_TABLE table=test-table [id=83b0596ba79c427abbde65e26eb1c57f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:31:34.927801 30814 tablet_service.cc:1467] Processing CreateTablet for tablet 15d71270c9134dbbad7b931135edb7df (DEFAULT_TABLE table=test-table [id=83b0596ba79c427abbde65e26eb1c57f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:31:34.929224 30814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15d71270c9134dbbad7b931135edb7df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:34.929224 30683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15d71270c9134dbbad7b931135edb7df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:34.931022 30944 tablet_service.cc:1467] Processing CreateTablet for tablet 15d71270c9134dbbad7b931135edb7df (DEFAULT_TABLE table=test-table [id=83b0596ba79c427abbde65e26eb1c57f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:31:34.932385 30944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15d71270c9134dbbad7b931135edb7df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:34.944414 31028 tablet_bootstrap.cc:492] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33: Bootstrap starting.
I20260925 09:31:34.948668 31027 tablet_bootstrap.cc:492] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785: Bootstrap starting.
I20260925 09:31:34.950383 31028 tablet_bootstrap.cc:654] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:34.951301 31029 tablet_bootstrap.cc:492] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b: Bootstrap starting.
I20260925 09:31:34.951843 31028 log.cc:824] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:34.953075 31027 tablet_bootstrap.cc:654] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:34.954037 31027 log.cc:824] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:34.954957 31029 tablet_bootstrap.cc:654] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:34.955135 31028 tablet_bootstrap.cc:492] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33: No bootstrap required, opened a new log
I20260925 09:31:34.955441 31028 ts_tablet_manager.cc:1397] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.002s
I20260925 09:31:34.956323 31029 log.cc:824] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:34.956785 31027 tablet_bootstrap.cc:492] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785: No bootstrap required, opened a new log
I20260925 09:31:34.957132 31027 ts_tablet_manager.cc:1397] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260925 09:31:34.959328 31029 tablet_bootstrap.cc:492] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b: No bootstrap required, opened a new log
I20260925 09:31:34.959640 31029 ts_tablet_manager.cc:1397] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260925 09:31:34.965474 31027 raft_consensus.cc:348] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499978cf17764a0b9f4e1afbeb13ff1b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34807 } } peers { permanent_uuid: "e3fb93bfcedf425e8aa87768a8190b33" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 35141 } } peers { permanent_uuid: "37b530c91bd540f6bbc5c660f1392785" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 37705 } }
I20260925 09:31:34.966024 31027 raft_consensus.cc:374] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:34.966179 31027 raft_consensus.cc:729] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37b530c91bd540f6bbc5c660f1392785, State: Initialized, Role: FOLLOWER
I20260925 09:31:34.966889 31027 consensus_queue.cc:260] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499978cf17764a0b9f4e1afbeb13ff1b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34807 } } peers { permanent_uuid: "e3fb93bfcedf425e8aa87768a8190b33" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 35141 } } peers { permanent_uuid: "37b530c91bd540f6bbc5c660f1392785" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 37705 } }
I20260925 09:31:34.967211 31028 raft_consensus.cc:348] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499978cf17764a0b9f4e1afbeb13ff1b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34807 } } peers { permanent_uuid: "e3fb93bfcedf425e8aa87768a8190b33" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 35141 } } peers { permanent_uuid: "37b530c91bd540f6bbc5c660f1392785" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 37705 } }
I20260925 09:31:34.967840 31028 raft_consensus.cc:374] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:34.967995 31028 raft_consensus.cc:729] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3fb93bfcedf425e8aa87768a8190b33, State: Initialized, Role: FOLLOWER
I20260925 09:31:34.968722 31028 consensus_queue.cc:260] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499978cf17764a0b9f4e1afbeb13ff1b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34807 } } peers { permanent_uuid: "e3fb93bfcedf425e8aa87768a8190b33" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 35141 } } peers { permanent_uuid: "37b530c91bd540f6bbc5c660f1392785" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 37705 } }
I20260925 09:31:34.974465 31029 raft_consensus.cc:348] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499978cf17764a0b9f4e1afbeb13ff1b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34807 } } peers { permanent_uuid: "e3fb93bfcedf425e8aa87768a8190b33" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 35141 } } peers { permanent_uuid: "37b530c91bd540f6bbc5c660f1392785" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 37705 } }
I20260925 09:31:34.975042 31029 raft_consensus.cc:374] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:34.975286 31029 raft_consensus.cc:729] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499978cf17764a0b9f4e1afbeb13ff1b, State: Initialized, Role: FOLLOWER
I20260925 09:31:34.976102 31029 consensus_queue.cc:260] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499978cf17764a0b9f4e1afbeb13ff1b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34807 } } peers { permanent_uuid: "e3fb93bfcedf425e8aa87768a8190b33" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 35141 } } peers { permanent_uuid: "37b530c91bd540f6bbc5c660f1392785" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 37705 } }
I20260925 09:31:34.977469 31028 ts_tablet_manager.cc:1428] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33: Time spent starting tablet: real 0.022s user 0.017s sys 0.004s
I20260925 09:31:34.977819 30879 heartbeater.cc:503] Master 127.10.114.126:39781 was elected leader, sending a full tablet report...
I20260925 09:31:34.983621 31009 heartbeater.cc:503] Master 127.10.114.126:39781 was elected leader, sending a full tablet report...
I20260925 09:31:34.984615 30748 heartbeater.cc:503] Master 127.10.114.126:39781 was elected leader, sending a full tablet report...
I20260925 09:31:34.985154 31027 ts_tablet_manager.cc:1428] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785: Time spent starting tablet: real 0.028s user 0.023s sys 0.004s
I20260925 09:31:34.985172 31029 ts_tablet_manager.cc:1428] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b: Time spent starting tablet: real 0.025s user 0.022s sys 0.000s
W20260925 09:31:35.102640 31010 tablet.cc:2369] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:31:35.154718 30880 tablet.cc:2369] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:31:35.169893 31035 raft_consensus.cc:482] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:31:35.170161 31035 raft_consensus.cc:504] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499978cf17764a0b9f4e1afbeb13ff1b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34807 } } peers { permanent_uuid: "e3fb93bfcedf425e8aa87768a8190b33" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 35141 } } peers { permanent_uuid: "37b530c91bd540f6bbc5c660f1392785" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 37705 } }
I20260925 09:31:35.171751 31035 leader_election.cc:288] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3fb93bfcedf425e8aa87768a8190b33 (127.10.114.66:35141), 37b530c91bd540f6bbc5c660f1392785 (127.10.114.67:37705)
I20260925 09:31:35.179443 31024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:35.172286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:41971 (local address 127.10.114.66:35141)
0925 09:31:35.172514 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:35.172523 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:35.173019 (+ 496us) server_negotiation.cc:407] Connection header received
0925 09:31:35.173305 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:35.173313 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:35.173420 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:35.173504 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:35.174366 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:35.175219 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:35.177180 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:35.178307 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:35.178478 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:35.178565 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:35.179075 (+ 510us) server_negotiation.cc:299] Negotiation successful
0925 09:31:35.179192 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:31:35.179514 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:35.173217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.114.65:56847 (local address 127.10.114.67:37705)
0925 09:31:35.173414 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:35.173420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:35.173670 (+ 250us) server_negotiation.cc:407] Connection header received
0925 09:31:35.173792 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:35.173798 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:35.173878 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:35.173977 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:31:35.174778 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:35.175724 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:35.177435 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:35.178562 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:35.178709 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:35.178782 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:35.179165 (+ 383us) server_negotiation.cc:299] Negotiation successful
0925 09:31:35.179280 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 09:31:35.179584 31036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:35.172205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.66:35141 (local address 127.10.114.65:41971)
0925 09:31:35.172866 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:35.172912 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:35.173068 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:35.173704 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:35.173709 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:35.173741 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:35.174177 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:35.174193 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:35.175354 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:35.175359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:35.176997 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:35.177009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:35.178507 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:35.178542 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:35.178771 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:35.178840 (+ 69us) client_negotiation.cc:769] Sending connection context
0925 09:31:35.179042 (+ 202us) client_negotiation.cc:240] Negotiation successful
0925 09:31:35.179166 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":2,"thread_start_us":380,"threads_started":1}
I20260925 09:31:35.179590 31037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:35.172957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.67:37705 (local address 127.10.114.65:56847)
0925 09:31:35.173564 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:35.173578 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 09:31:35.173681 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:31:35.174004 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:31:35.174007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:31:35.174030 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:31:35.174551 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:35.174563 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:35.175866 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:35.175870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:35.177261 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:31:35.177274 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:31:35.178766 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:31:35.178774 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:31:35.178955 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:35.179004 (+ 49us) client_negotiation.cc:769] Sending connection context
0925 09:31:35.179139 (+ 135us) client_negotiation.cc:240] Negotiation successful
0925 09:31:35.179265 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":292,"threads_started":1}
I20260925 09:31:35.180497 30834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15d71270c9134dbbad7b931135edb7df" candidate_uuid: "499978cf17764a0b9f4e1afbeb13ff1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3fb93bfcedf425e8aa87768a8190b33" is_pre_election: true
I20260925 09:31:35.180857 30964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15d71270c9134dbbad7b931135edb7df" candidate_uuid: "499978cf17764a0b9f4e1afbeb13ff1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37b530c91bd540f6bbc5c660f1392785" is_pre_election: true
I20260925 09:31:35.180955 30834 raft_consensus.cc:2454] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 499978cf17764a0b9f4e1afbeb13ff1b in term 0.
I20260925 09:31:35.181325 30964 raft_consensus.cc:2454] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 499978cf17764a0b9f4e1afbeb13ff1b in term 0.
I20260925 09:31:35.181584 30638 leader_election.cc:302] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 499978cf17764a0b9f4e1afbeb13ff1b, e3fb93bfcedf425e8aa87768a8190b33; no voters:
I20260925 09:31:35.181931 31035 raft_consensus.cc:2781] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:31:35.182067 31035 raft_consensus.cc:482] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:31:35.182168 31035 raft_consensus.cc:3037] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:35.184252 31035 raft_consensus.cc:504] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499978cf17764a0b9f4e1afbeb13ff1b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34807 } } peers { permanent_uuid: "e3fb93bfcedf425e8aa87768a8190b33" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 35141 } } peers { permanent_uuid: "37b530c91bd540f6bbc5c660f1392785" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 37705 } }
I20260925 09:31:35.184803 31035 leader_election.cc:288] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [CANDIDATE]: Term 1 election: Requested vote from peers e3fb93bfcedf425e8aa87768a8190b33 (127.10.114.66:35141), 37b530c91bd540f6bbc5c660f1392785 (127.10.114.67:37705)
I20260925 09:31:35.185186 30834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15d71270c9134dbbad7b931135edb7df" candidate_uuid: "499978cf17764a0b9f4e1afbeb13ff1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3fb93bfcedf425e8aa87768a8190b33"
I20260925 09:31:35.185299 30964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15d71270c9134dbbad7b931135edb7df" candidate_uuid: "499978cf17764a0b9f4e1afbeb13ff1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37b530c91bd540f6bbc5c660f1392785"
I20260925 09:31:35.185353 30834 raft_consensus.cc:3037] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:35.185483 30964 raft_consensus.cc:3037] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:35.187271 30834 raft_consensus.cc:2454] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 499978cf17764a0b9f4e1afbeb13ff1b in term 1.
I20260925 09:31:35.187459 30964 raft_consensus.cc:2454] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 499978cf17764a0b9f4e1afbeb13ff1b in term 1.
I20260925 09:31:35.187683 30638 leader_election.cc:302] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 499978cf17764a0b9f4e1afbeb13ff1b, e3fb93bfcedf425e8aa87768a8190b33; no voters:
I20260925 09:31:35.187968 31035 raft_consensus.cc:2781] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:35.188380 31035 raft_consensus.cc:686] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [term 1 LEADER]: Becoming Leader. State: Replica: 499978cf17764a0b9f4e1afbeb13ff1b, State: Running, Role: LEADER
I20260925 09:31:35.188830 31035 consensus_queue.cc:237] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499978cf17764a0b9f4e1afbeb13ff1b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34807 } } peers { permanent_uuid: "e3fb93bfcedf425e8aa87768a8190b33" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 35141 } } peers { permanent_uuid: "37b530c91bd540f6bbc5c660f1392785" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 37705 } }
I20260925 09:31:35.193332 30561 catalog_manager.cc:5495] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b reported cstate change: term changed from 0 to 1, leader changed from <none> to 499978cf17764a0b9f4e1afbeb13ff1b (127.10.114.65). New cstate: current_term: 1 leader_uuid: "499978cf17764a0b9f4e1afbeb13ff1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499978cf17764a0b9f4e1afbeb13ff1b" member_type: VOTER last_known_addr { host: "127.10.114.65" port: 34807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e3fb93bfcedf425e8aa87768a8190b33" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 35141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37b530c91bd540f6bbc5c660f1392785" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 37705 } health_report { overall_health: UNKNOWN } } }
I20260925 09:31:35.214650 31022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:35.209305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43750 (local address 127.10.114.65:34807)
0925 09:31:35.209508 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:35.209514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:35.209534 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:31:35.209619 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:35.209623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:35.209719 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:35.209873 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:31:35.210234 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:35.210921 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:35.212426 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:35.212935 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:35.213113 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:35.213591 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:31:35.214074 (+ 483us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:31:35.214243 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:35.214315 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 09:31:35.214390 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260925 09:31:35.216423 30749 tablet.cc:2369] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:31:35.223618 30834 raft_consensus.cc:1261] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33 [term 1 FOLLOWER]: Refusing update from remote peer 499978cf17764a0b9f4e1afbeb13ff1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:31:35.223618 30964 raft_consensus.cc:1261] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785 [term 1 FOLLOWER]: Refusing update from remote peer 499978cf17764a0b9f4e1afbeb13ff1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:31:35.224659 31038 consensus_queue.cc:1035] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3fb93bfcedf425e8aa87768a8190b33" member_type: VOTER last_known_addr { host: "127.10.114.66" port: 35141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:31:35.224999 31035 consensus_queue.cc:1035] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b530c91bd540f6bbc5c660f1392785" member_type: VOTER last_known_addr { host: "127.10.114.67" port: 37705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:31:35.235090 31041 mvcc.cc:205] Tried to move back new op lower bound from 7333186335625400320 to 7333186335496916992. Current Snapshot: MvccSnapshot[applied={T|T < 7333186335625400320}]
I20260925 09:31:35.238534 31043 mvcc.cc:205] Tried to move back new op lower bound from 7333186335625400320 to 7333186335496916992. Current Snapshot: MvccSnapshot[applied={T|T < 7333186335625400320}]
I20260925 09:31:35.248348 30561 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:31:35.248574 30561 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:31:35.251760 30561 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d878527386fa43398a7c1909e7ddaaa5: Sending DeleteTablet for 3 replicas of tablet 15d71270c9134dbbad7b931135edb7df
I20260925 09:31:35.252785 30683 tablet_service.cc:1514] Processing DeleteTablet for tablet 15d71270c9134dbbad7b931135edb7df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:31:35 UTC) from {username='slave'} at 127.0.0.1:43742
I20260925 09:31:35.253095 30944 tablet_service.cc:1514] Processing DeleteTablet for tablet 15d71270c9134dbbad7b931135edb7df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:31:35 UTC) from {username='slave'} at 127.0.0.1:51502
I20260925 09:31:35.253742 31050 tablet_replica.cc:330] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b: stopping tablet replica
I20260925 09:31:35.253993 30814 tablet_service.cc:1514] Processing DeleteTablet for tablet 15d71270c9134dbbad7b931135edb7df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:31:35 UTC) from {username='slave'} at 127.0.0.1:48124
I20260925 09:31:35.254254 31050 raft_consensus.cc:2229] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:31:35.254585 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 30621
I20260925 09:31:35.255008 31050 raft_consensus.cc:2258] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:31:35.258950 31050 ts_tablet_manager.cc:1905] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:31:35.264246 31022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:35.255631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43760 (local address 127.10.114.65:34807)
0925 09:31:35.255816 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:35.255825 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:35.256145 (+ 320us) server_negotiation.cc:407] Connection header received
0925 09:31:35.256382 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:35.256387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:35.256469 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:35.256537 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:35.257146 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:35.258317 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:35.260250 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:35.260930 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:35.261028 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:35.262465 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:35.262474 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:35.262489 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:35.263234 (+ 745us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:35.263288 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:35.263313 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:35.263400 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:35.263646 (+ 246us) server_negotiation.cc:299] Negotiation successful
0925 09:31:35.263832 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 09:31:35.265529 30743 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:35.266848 31051 tablet_replica.cc:330] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785: stopping tablet replica
I20260925 09:31:35.267303 31051 raft_consensus.cc:2229] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:31:35.267838 31051 raft_consensus.cc:2258] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:31:35.269814 31051 ts_tablet_manager.cc:1905] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:31:35.270589 31052 tablet_replica.cc:330] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33: stopping tablet replica
I20260925 09:31:35.270911 31052 raft_consensus.cc:2229] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:31:35.271296 31052 raft_consensus.cc:2258] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:31:35.272845 31052 ts_tablet_manager.cc:1905] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:31:35.277104 31051 ts_tablet_manager.cc:1918] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:31:35.277292 31051 log.cc:1196] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-2/wal/wals/15d71270c9134dbbad7b931135edb7df
I20260925 09:31:35.277338 31052 ts_tablet_manager.cc:1918] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:31:35.277495 31052 log.cc:1196] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-1/wal/wals/15d71270c9134dbbad7b931135edb7df
I20260925 09:31:35.277887 31051 ts_tablet_manager.cc:1939] T 15d71270c9134dbbad7b931135edb7df P 37b530c91bd540f6bbc5c660f1392785: Deleting consensus metadata
I20260925 09:31:35.278007 31052 ts_tablet_manager.cc:1939] T 15d71270c9134dbbad7b931135edb7df P e3fb93bfcedf425e8aa87768a8190b33: Deleting consensus metadata
I20260925 09:31:35.279475 30548 catalog_manager.cc:4841] TS e3fb93bfcedf425e8aa87768a8190b33 (127.10.114.66:35141): tablet 15d71270c9134dbbad7b931135edb7df (table test-table [id=83b0596ba79c427abbde65e26eb1c57f]) successfully deleted
I20260925 09:31:35.279850 30548 catalog_manager.cc:4841] TS 37b530c91bd540f6bbc5c660f1392785 (127.10.114.67:37705): tablet 15d71270c9134dbbad7b931135edb7df (table test-table [id=83b0596ba79c427abbde65e26eb1c57f]) successfully deleted
I20260925 09:31:35.362890 31050 ts_tablet_manager.cc:1918] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:31:35.363150 31050 log.cc:1196] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b: Deleting WAL directory at /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790328385296020-10697-0/minicluster-data/ts-0/wal/wals/15d71270c9134dbbad7b931135edb7df
I20260925 09:31:35.363608 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 30621
I20260925 09:31:35.363765 31050 ts_tablet_manager.cc:1939] T 15d71270c9134dbbad7b931135edb7df P 499978cf17764a0b9f4e1afbeb13ff1b: Deleting consensus metadata
I20260925 09:31:35.382164 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 30752
W20260925 09:31:35.382189 30549 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.114.65:34807 (error 108)
I20260925 09:31:35.384059 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:35.383632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:34807 (local address 127.0.0.1:43770)
0925 09:31:35.383807 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:35.383927 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:34807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260925 09:31:35.384270 30549 catalog_manager.cc:4584] TS 499978cf17764a0b9f4e1afbeb13ff1b (127.10.114.65:34807): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 15d71270c9134dbbad7b931135edb7df: Network error: Client connection negotiation failed: client connection to 127.10.114.65:34807: connect: Connection refused (error 111)
I20260925 09:31:35.388958 31024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:35.383235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48128 (local address 127.10.114.66:35141)
0925 09:31:35.383453 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:35.383461 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:35.383481 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:31:35.383549 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:35.383555 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:35.383684 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:35.383848 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:35.384301 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:35.385075 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:35.386300 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:35.386669 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:35.386760 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:35.387729 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:35.387737 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:35.387752 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:35.388396 (+ 644us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:35.388438 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:35.388455 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:35.388539 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:35.388654 (+ 115us) server_negotiation.cc:299] Negotiation successful
0925 09:31:35.388731 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 09:31:35.389635 30874 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:35.437566 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:35.437103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:34807 (local address 127.0.0.1:43780)
0925 09:31:35.437317 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:35.437415 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:34807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260925 09:31:35.438537 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:35.438180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:34807 (local address 127.0.0.1:43784)
0925 09:31:35.438339 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:35.438415 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:34807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260925 09:31:35.475196 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 30752
I20260925 09:31:35.492144 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 30882
I20260925 09:31:35.498911 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:35.492956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51514 (local address 127.10.114.67:37705)
0925 09:31:35.493129 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:35.493134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:35.493167 (+ 33us) server_negotiation.cc:407] Connection header received
0925 09:31:35.493227 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:35.493233 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:35.493321 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:35.493431 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:35.493848 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:35.494557 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:35.495953 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:35.496441 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:35.496574 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:35.497492 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:35.497501 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:35.497514 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:35.498163 (+ 649us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:35.498218 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:35.498240 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:35.498350 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:35.498490 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 09:31:35.498594 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 09:31:35.499135 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:35.498616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:34807 (local address 127.0.0.1:43790)
0925 09:31:35.498821 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:35.498895 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:34807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260925 09:31:35.499656 31004 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:35.500205 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:35.499864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:34807 (local address 127.0.0.1:43792)
0925 09:31:35.500007 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:35.500097 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:34807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 09:31:35.580860 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:35.580292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:34807 (local address 127.0.0.1:43806)
0925 09:31:35.580539 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:35.580649 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:34807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260925 09:31:35.581938 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:35.581516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.65:34807 (local address 127.0.0.1:43818)
0925 09:31:35.581727 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0925 09:31:35.581819 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.65:34807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260925 09:31:35.586815 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 30882
I20260925 09:31:35.602985 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 30531
I20260925 09:31:35.604182 30592 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:35.711632 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 30531
2026-09-25T09:31:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2809 ms)
[----------] 6 tests from EncryptionPolicyTest (20919 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:31:35 dist-test-slave-9pj8 krb5kdc[31061](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:31:35 dist-test-slave-9pj8 krb5kdc[31061](info): set up 2 sockets
Sep 25 09:31:35 dist-test-slave-9pj8 krb5kdc[31061](info): commencing operation
krb5kdc: starting...
W20260925 09:31:36.808521 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:31:36 dist-test-slave-9pj8 krb5kdc[31061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:31:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:31:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:31:37.044955 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33235
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:37.366361 31074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:37.366613 31074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:37.366699 31074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:37.374346 31074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:31:37.374435 31074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:37.374478 31074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:37.374534 31074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:31:37.374578 31074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:31:37.383760 31074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33235
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:37.384936 31074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:37.386123 31074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:37.397132 31082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:37.397102 31080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:37.397614 31079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:37.398522 31074 server_base.cc:1034] running on GCE node
I20260925 09:31:37.399284 31074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:37.400712 31074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:37.401914 31074 hybrid_clock.cc:648] HybridClock initialized: now 1790328697401871 us; error 49 us; skew 500 ppm
I20260925 09:31:37.402245 31074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:37 dist-test-slave-9pj8 krb5kdc[31061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328697, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:37.405391 31074 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:31:37.406793 31074 webserver.cc:460] Webserver started at http://127.0.0.1:37093/ using document root <none> and password file <none>
I20260925 09:31:37.407445 31074 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:37.407574 31074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:37.407944 31074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:37.410774 31074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "80c4a6fe469e4323b67222c886ea4eeb"
format_stamp: "Formatted at 2026-09-25 09:31:37 on dist-test-slave-9pj8"
server_key: "54048f5ec9cffadbbea8f3be0be1f384"
server_key_iv: "1632a20fa7b17bf553846341f8796e3d"
server_key_version: "encryptionkey@0"
I20260925 09:31:37.411530 31074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "80c4a6fe469e4323b67222c886ea4eeb"
format_stamp: "Formatted at 2026-09-25 09:31:37 on dist-test-slave-9pj8"
server_key: "54048f5ec9cffadbbea8f3be0be1f384"
server_key_iv: "1632a20fa7b17bf553846341f8796e3d"
server_key_version: "encryptionkey@0"
I20260925 09:31:37.416455 31074 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260925 09:31:37.419899 31089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:37.421182 31074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:31:37.421411 31074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "80c4a6fe469e4323b67222c886ea4eeb"
format_stamp: "Formatted at 2026-09-25 09:31:37 on dist-test-slave-9pj8"
server_key: "54048f5ec9cffadbbea8f3be0be1f384"
server_key_iv: "1632a20fa7b17bf553846341f8796e3d"
server_key_version: "encryptionkey@0"
I20260925 09:31:37.421613 31074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:37.433812 31074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:37.437793 31074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:37.438062 31074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:37.456282 31074 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33235
I20260925 09:31:37.456317 31140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33235 every 8 connection(s)
I20260925 09:31:37.457614 31074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:31:37.461345 31141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:37.465605 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 31074
I20260925 09:31:37.465829 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:31:37.466715 10697 external_mini_cluster.cc:1428] Setting key 7e2ea574e3e5d0f19482d99421cbd9ae
I20260925 09:31:37.475345 31141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb: Bootstrap starting.
I20260925 09:31:37.479432 31141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:37.480751 31141 log.cc:824] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb: Log is configured to *not* fsync() on all Append() calls
Sep 25 09:31:37 dist-test-slave-9pj8 krb5kdc[31061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 09:31:37.483477 31141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb: No bootstrap required, opened a new log
I20260925 09:31:37.491606 31144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:37.468665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34313 (local address 127.0.0.1:33235)
0925 09:31:37.469418 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:37.469429 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:37.469467 (+ 38us) server_negotiation.cc:407] Connection header received
0925 09:31:37.470995 (+ 1528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:37.471014 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:37.471397 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:37.471730 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:37.473620 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:37.474890 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:37.476857 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:37.477704 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:37.477860 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:37.482752 (+ 4892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:37.482767 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:37.482782 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:37.482823 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:37.486871 (+ 4048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:37.487735 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:37.487743 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:37.487751 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:37.488002 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:37.488510 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:37.488515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:37.488519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:37.489216 (+ 697us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:37.489576 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:37.489912 (+ 336us) server_negotiation.cc:299] Negotiation successful
0925 09:31:37.490246 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":379,"threads_started":1}
I20260925 09:31:37.494038 31141 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c4a6fe469e4323b67222c886ea4eeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33235 } }
I20260925 09:31:37.494437 31141 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:37.494549 31141 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80c4a6fe469e4323b67222c886ea4eeb, State: Initialized, Role: FOLLOWER
I20260925 09:31:37.495177 31141 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c4a6fe469e4323b67222c886ea4eeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33235 } }
I20260925 09:31:37.495466 31141 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:31:37.495615 31141 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:31:37.495832 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:31:37.495844 31141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:37.497916 31141 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c4a6fe469e4323b67222c886ea4eeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33235 } }
I20260925 09:31:37.498385 31141 leader_election.cc:302] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80c4a6fe469e4323b67222c886ea4eeb; no voters:
I20260925 09:31:37.499075 31141 leader_election.cc:288] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:31:37.499171 31148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:37.499811 31148 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb [term 1 LEADER]: Becoming Leader. State: Replica: 80c4a6fe469e4323b67222c886ea4eeb, State: Running, Role: LEADER
I20260925 09:31:37.500367 31148 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c4a6fe469e4323b67222c886ea4eeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33235 } }
I20260925 09:31:37.501976 31141 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:31:37.504015 31152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80c4a6fe469e4323b67222c886ea4eeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c4a6fe469e4323b67222c886ea4eeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33235 } } }
I20260925 09:31:37.504767 31152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:37.504532 31151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80c4a6fe469e4323b67222c886ea4eeb. Latest consensus state: current_term: 1 leader_uuid: "80c4a6fe469e4323b67222c886ea4eeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c4a6fe469e4323b67222c886ea4eeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33235 } } }
I20260925 09:31:37.504995 31151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:37.506080 31157 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:31:37.510836 31157 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:31:37.522518 31144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:37.501187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49583 (local address 127.0.0.1:33235)
0925 09:31:37.501350 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:37.501355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:37.501373 (+ 18us) server_negotiation.cc:407] Connection header received
0925 09:31:37.501543 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:37.501550 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:37.501629 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:37.501704 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:37.504012 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:37.505352 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:37.507622 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:37.508582 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:37.508725 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:37.512891 (+ 4166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:37.512900 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:37.512907 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:37.512955 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:37.517335 (+ 4380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:37.518559 (+ 1224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:37.518567 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:37.518573 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:37.518745 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:37.519445 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:37.519453 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:37.519458 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:37.519991 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:37.520300 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:37.521786 (+ 1486us) server_negotiation.cc:299] Negotiation successful
0925 09:31:37.522017 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 09:31:37.527246 31157 catalog_manager.cc:1347] Generated new cluster ID: a97c93baf84c4e1e8e87e76c06815d12
I20260925 09:31:37.527338 31157 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:31:37.544840 31157 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:31:37.546108 31157 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:31:37.557462 31157 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80c4a6fe469e4323b67222c886ea4eeb: Generated new TSK 0
I20260925 09:31:37.558177 31157 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:31:37.564466 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 31074
I20260925 09:31:37.566296 31136 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:37.654160 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 31074
W20260925 09:31:37.671213 31059 connection.cc:535] client connection to 127.0.0.1:33235 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33235): Connection reset by peer (error 104)
2026-09-25T09:31:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1932 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:31:37 dist-test-slave-9pj8 krb5kdc[31173](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:31:37 dist-test-slave-9pj8 krb5kdc[31173](info): set up 2 sockets
Sep 25 09:31:37 dist-test-slave-9pj8 krb5kdc[31173](info): commencing operation
krb5kdc: starting...
W20260925 09:31:38.732591 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:31:38 dist-test-slave-9pj8 krb5kdc[31173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:31:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:31:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:31:38.978654 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37777
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:39.298405 31186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:39.298661 31186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:39.298748 31186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:39.305408 31186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:31:39.305496 31186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:39.305539 31186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:39.305596 31186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:31:39.305639 31186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:31:39.314568 31186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37777
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:39.315672 31186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:39.317338 31186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:39.328733 31194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:39.328830 31191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:39.328827 31192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:39.329768 31186 server_base.cc:1034] running on GCE node
I20260925 09:31:39.330482 31186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:39.331761 31186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:39.333021 31186 hybrid_clock.cc:648] HybridClock initialized: now 1790328699332988 us; error 28 us; skew 500 ppm
I20260925 09:31:39.333308 31186 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:39 dist-test-slave-9pj8 krb5kdc[31173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328699, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:39.340224 31186 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:31:39.341748 31186 webserver.cc:460] Webserver started at http://127.0.0.1:44589/ using document root <none> and password file <none>
I20260925 09:31:39.342427 31186 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:39.342523 31186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:39.342850 31186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:39.345829 31186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "e720b0f261bb40ff8c95a12cdeeb4211"
format_stamp: "Formatted at 2026-09-25 09:31:39 on dist-test-slave-9pj8"
server_key: "3d0166aa8f7705c3ad1a549f76857297"
server_key_iv: "45ae5d26a04186fac5d6b3e98584e762"
server_key_version: "encryptionkey@0"
I20260925 09:31:39.346558 31186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "e720b0f261bb40ff8c95a12cdeeb4211"
format_stamp: "Formatted at 2026-09-25 09:31:39 on dist-test-slave-9pj8"
server_key: "3d0166aa8f7705c3ad1a549f76857297"
server_key_iv: "45ae5d26a04186fac5d6b3e98584e762"
server_key_version: "encryptionkey@0"
I20260925 09:31:39.351492 31186 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260925 09:31:39.354826 31201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:39.356149 31186 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:31:39.356338 31186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "e720b0f261bb40ff8c95a12cdeeb4211"
format_stamp: "Formatted at 2026-09-25 09:31:39 on dist-test-slave-9pj8"
server_key: "3d0166aa8f7705c3ad1a549f76857297"
server_key_iv: "45ae5d26a04186fac5d6b3e98584e762"
server_key_version: "encryptionkey@0"
I20260925 09:31:39.356524 31186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:39.374231 31186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:39.378228 31186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:39.378496 31186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:39.396754 31186 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37777
I20260925 09:31:39.396772 31252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37777 every 8 connection(s)
I20260925 09:31:39.397950 31186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:31:39.401057 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 31186
I20260925 09:31:39.401253 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:31:39.401856 10697 external_mini_cluster.cc:1428] Setting key 172b4c80a55d2fe987307eb55caf58bd
I20260925 09:31:39.402053 31253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:39.414530 31253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211: Bootstrap starting.
I20260925 09:31:39.414697 31256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:39.403639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37721 (local address 127.0.0.1:37777)
0925 09:31:39.404423 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:39.404435 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:39.404474 (+ 39us) server_negotiation.cc:407] Connection header received
0925 09:31:39.406717 (+ 2243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:39.406736 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:39.407110 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:39.407477 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:39.408035 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:39.409298 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:39.410520 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:39.411170 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:39.411381 (+ 211us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:39.412645 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:39.412655 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:39.412667 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:39.413157 (+ 490us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:39.413227 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:39.413249 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:39.413406 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:39.413669 (+ 263us) server_negotiation.cc:299] Negotiation successful
0925 09:31:39.413822 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":340,"threads_started":1}
I20260925 09:31:39.417847 31253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:39.418210 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:31:39.419010 31253 log.cc:824] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:39.421794 31253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211: No bootstrap required, opened a new log
I20260925 09:31:39.428524 31256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:39.422488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42099 (local address 127.0.0.1:37777)
0925 09:31:39.422732 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:39.422740 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:39.423133 (+ 393us) server_negotiation.cc:407] Connection header received
0925 09:31:39.423331 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:39.423336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:39.423397 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:39.423508 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:39.424036 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:39.424823 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:39.426102 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:39.426530 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:39.426665 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:39.427720 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:39.427724 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:39.427727 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:39.427741 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:39.427853 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:39.427873 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:39.428005 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:39.428207 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 09:31:39.428255 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260925 09:31:39.430189 31253 raft_consensus.cc:348] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e720b0f261bb40ff8c95a12cdeeb4211" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37777 } }
I20260925 09:31:39.430593 31253 raft_consensus.cc:374] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:39.430716 31253 raft_consensus.cc:729] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e720b0f261bb40ff8c95a12cdeeb4211, State: Initialized, Role: FOLLOWER
I20260925 09:31:39.431309 31253 consensus_queue.cc:260] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e720b0f261bb40ff8c95a12cdeeb4211" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37777 } }
I20260925 09:31:39.431569 31253 raft_consensus.cc:388] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:31:39.431711 31253 raft_consensus.cc:482] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:31:39.431917 31253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:39.434422 31253 raft_consensus.cc:504] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e720b0f261bb40ff8c95a12cdeeb4211" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37777 } }
I20260925 09:31:39.434937 31253 leader_election.cc:302] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e720b0f261bb40ff8c95a12cdeeb4211; no voters:
I20260925 09:31:39.435570 31253 leader_election.cc:288] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:31:39.435662 31263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:39.436476 31263 raft_consensus.cc:686] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211 [term 1 LEADER]: Becoming Leader. State: Replica: e720b0f261bb40ff8c95a12cdeeb4211, State: Running, Role: LEADER
I20260925 09:31:39.437004 31263 consensus_queue.cc:237] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e720b0f261bb40ff8c95a12cdeeb4211" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37777 } }
I20260925 09:31:39.437873 31253 sys_catalog.cc:564] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:31:39.439791 31264 sys_catalog.cc:455] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e720b0f261bb40ff8c95a12cdeeb4211" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e720b0f261bb40ff8c95a12cdeeb4211" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37777 } } }
I20260925 09:31:39.440572 31264 sys_catalog.cc:458] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:39.439622 31265 sys_catalog.cc:455] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e720b0f261bb40ff8c95a12cdeeb4211. Latest consensus state: current_term: 1 leader_uuid: "e720b0f261bb40ff8c95a12cdeeb4211" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e720b0f261bb40ff8c95a12cdeeb4211" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37777 } } }
I20260925 09:31:39.441111 31265 sys_catalog.cc:458] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:39.442463 31268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:31:39.447044 31268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:31:39.456749 31268 catalog_manager.cc:1347] Generated new cluster ID: 203b2123b5914e919c43de7a23053fc9
I20260925 09:31:39.456866 31268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:31:39.468887 31268 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:31:39.470536 31268 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:31:39.480690 31268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e720b0f261bb40ff8c95a12cdeeb4211: Generated new TSK 0
I20260925 09:31:39.481647 31268 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:31:39.488121 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 31186
W20260925 09:31:39.489539 31248 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37721
W20260925 09:31:39.490283 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 31186: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:31:39.490478 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 31186
W20260925 09:31:39.506860 31171 connection.cc:535] client connection to 127.0.0.1:37777 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37777): Connection reset by peer (error 104)
2026-09-25T09:31:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1835 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:31:39 dist-test-slave-9pj8 krb5kdc[31284](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:31:39 dist-test-slave-9pj8 krb5kdc[31284](info): set up 2 sockets
Sep 25 09:31:39 dist-test-slave-9pj8 krb5kdc[31284](info): commencing operation
krb5kdc: starting...
W20260925 09:31:40.592716 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:31:40 dist-test-slave-9pj8 krb5kdc[31284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:31:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:31:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:31:40.828567 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44191
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:41.147454 31297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:41.147718 31297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:41.147884 31297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:41.154844 31297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:31:41.154939 31297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:41.154985 31297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:41.155036 31297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:31:41.155081 31297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:31:41.164678 31297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44191
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:41.165798 31297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:41.166999 31297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:41.178539 31302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:41.178541 31305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:41.178521 31303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:41.179160 31297 server_base.cc:1034] running on GCE node
I20260925 09:31:41.180007 31297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:41.181330 31297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:41.182569 31297 hybrid_clock.cc:648] HybridClock initialized: now 1790328701182537 us; error 41 us; skew 500 ppm
I20260925 09:31:41.182924 31297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:41 dist-test-slave-9pj8 krb5kdc[31284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328701, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:41.186201 31297 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:31:41.187502 31297 webserver.cc:460] Webserver started at http://127.0.0.1:46815/ using document root <none> and password file <none>
I20260925 09:31:41.188166 31297 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:41.188274 31297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:41.188525 31297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:41.191191 31297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "32408e41d7cf4e748f42c2c10085f011"
format_stamp: "Formatted at 2026-09-25 09:31:41 on dist-test-slave-9pj8"
server_key: "44d50ff4832115a11d7402e54a3ddd1a"
server_key_iv: "d1faf360dd3b6be8bbf11afb1f7e3b0b"
server_key_version: "encryptionkey@0"
I20260925 09:31:41.191915 31297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "32408e41d7cf4e748f42c2c10085f011"
format_stamp: "Formatted at 2026-09-25 09:31:41 on dist-test-slave-9pj8"
server_key: "44d50ff4832115a11d7402e54a3ddd1a"
server_key_iv: "d1faf360dd3b6be8bbf11afb1f7e3b0b"
server_key_version: "encryptionkey@0"
I20260925 09:31:41.196527 31297 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260925 09:31:41.199728 31312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:41.201006 31297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:31:41.201140 31297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "32408e41d7cf4e748f42c2c10085f011"
format_stamp: "Formatted at 2026-09-25 09:31:41 on dist-test-slave-9pj8"
server_key: "44d50ff4832115a11d7402e54a3ddd1a"
server_key_iv: "d1faf360dd3b6be8bbf11afb1f7e3b0b"
server_key_version: "encryptionkey@0"
I20260925 09:31:41.201293 31297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:41.206311 31297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:41.210419 31297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:41.210654 31297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:41.228894 31297 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44191
I20260925 09:31:41.228937 31363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44191 every 8 connection(s)
I20260925 09:31:41.230309 31297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:31:41.234050 31364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:41.240159 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 31297
I20260925 09:31:41.240339 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:31:41.240767 10697 external_mini_cluster.cc:1428] Setting key 6eff25dea90b3f8b375e28cf6017f730
I20260925 09:31:41.244199 31364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011: Bootstrap starting.
I20260925 09:31:41.247653 31364 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:41.248498 31367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:41.242443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41799 (local address 127.0.0.1:44191)
0925 09:31:41.243217 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:41.243229 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:41.243269 (+ 40us) server_negotiation.cc:407] Connection header received
0925 09:31:41.245012 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:41.245033 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:41.245356 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:41.245689 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:41.246772 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:41.246778 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:41.246790 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:41.246821 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:41.246876 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:41.246895 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:41.247030 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:41.247198 (+ 168us) server_negotiation.cc:299] Negotiation successful
0925 09:31:41.247400 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":359,"threads_started":1}
I20260925 09:31:41.248955 31364 log.cc:824] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:41.251951 31364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011: No bootstrap required, opened a new log
I20260925 09:31:41.252288 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:31:41.259071 31367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:41.256541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44711 (local address 127.0.0.1:44191)
0925 09:31:41.256714 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:41.256721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:41.257200 (+ 479us) server_negotiation.cc:407] Connection header received
0925 09:31:41.257403 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:41.257407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:41.257455 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:41.257541 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:41.258401 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:41.258406 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:41.258412 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:41.258426 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:41.258462 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:41.258478 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:41.258567 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:41.258697 (+ 130us) server_negotiation.cc:299] Negotiation successful
0925 09:31:41.258739 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 09:31:41.260340 31364 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32408e41d7cf4e748f42c2c10085f011" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44191 } }
I20260925 09:31:41.260694 31364 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:41.260814 31364 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32408e41d7cf4e748f42c2c10085f011, State: Initialized, Role: FOLLOWER
I20260925 09:31:41.261390 31364 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32408e41d7cf4e748f42c2c10085f011" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44191 } }
I20260925 09:31:41.261629 31364 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:31:41.261763 31364 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:31:41.261953 31364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:41.264207 31364 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32408e41d7cf4e748f42c2c10085f011" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44191 } }
I20260925 09:31:41.264724 31364 leader_election.cc:302] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32408e41d7cf4e748f42c2c10085f011; no voters:
I20260925 09:31:41.265336 31364 leader_election.cc:288] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:31:41.265473 31374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:41.266121 31374 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011 [term 1 LEADER]: Becoming Leader. State: Replica: 32408e41d7cf4e748f42c2c10085f011, State: Running, Role: LEADER
I20260925 09:31:41.266623 31374 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32408e41d7cf4e748f42c2c10085f011" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44191 } }
I20260925 09:31:41.267372 31364 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:31:41.269879 31376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32408e41d7cf4e748f42c2c10085f011. Latest consensus state: current_term: 1 leader_uuid: "32408e41d7cf4e748f42c2c10085f011" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32408e41d7cf4e748f42c2c10085f011" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44191 } } }
I20260925 09:31:41.270469 31376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:41.270468 31375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32408e41d7cf4e748f42c2c10085f011" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32408e41d7cf4e748f42c2c10085f011" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44191 } } }
I20260925 09:31:41.272414 31375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:41.276772 31384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:31:41.282078 31384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:31:41.290387 31384 catalog_manager.cc:1347] Generated new cluster ID: cb5e8e96706345589438052be32074d7
I20260925 09:31:41.290567 31384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:31:41.309752 31384 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:31:41.310226 31384 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:31:41.316087 31384 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32408e41d7cf4e748f42c2c10085f011: Generated new TSK 0
I20260925 09:31:41.316825 31384 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:31:41.322355 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 31297
W20260925 09:31:41.323333 31359 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41799
W20260925 09:31:41.324002 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 31297: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:31:41.324160 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 31297
2026-09-25T09:31:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1833 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:31:41 dist-test-slave-9pj8 krb5kdc[31395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:31:41 dist-test-slave-9pj8 krb5kdc[31395](info): set up 2 sockets
Sep 25 09:31:41 dist-test-slave-9pj8 krb5kdc[31395](info): commencing operation
krb5kdc: starting...
W20260925 09:31:42.440107 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:31:42 dist-test-slave-9pj8 krb5kdc[31395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:31:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:31:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:31:42.688818 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46859
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:43.006044 31408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:43.006388 31408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:43.006552 31408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:43.013618 31408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:31:43.013763 31408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:43.013878 31408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:43.013989 31408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:31:43.014086 31408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:31:43.023650 31408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:43.024978 31408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:43.026276 31408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:43.037662 31414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:43.037752 31413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:43.037669 31416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:43.038550 31408 server_base.cc:1034] running on GCE node
I20260925 09:31:43.039261 31408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:43.040633 31408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:43.041836 31408 hybrid_clock.cc:648] HybridClock initialized: now 1790328703041790 us; error 40 us; skew 500 ppm
I20260925 09:31:43.042189 31408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:43 dist-test-slave-9pj8 krb5kdc[31395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328703, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:43.045272 31408 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:31:43.046728 31408 webserver.cc:460] Webserver started at http://127.0.0.1:41447/ using document root <none> and password file <none>
I20260925 09:31:43.047350 31408 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:43.047474 31408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:43.047761 31408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:43.050642 31408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "a467a0401fb24d69bfa3a101a3e66e2a"
format_stamp: "Formatted at 2026-09-25 09:31:43 on dist-test-slave-9pj8"
server_key: "af60ba78a7c3503f740fbd6ad657b082"
server_key_iv: "118d765458ec9be26875d68774bbbe09"
server_key_version: "encryptionkey@0"
I20260925 09:31:43.051398 31408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "a467a0401fb24d69bfa3a101a3e66e2a"
format_stamp: "Formatted at 2026-09-25 09:31:43 on dist-test-slave-9pj8"
server_key: "af60ba78a7c3503f740fbd6ad657b082"
server_key_iv: "118d765458ec9be26875d68774bbbe09"
server_key_version: "encryptionkey@0"
I20260925 09:31:43.056166 31408 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260925 09:31:43.059367 31423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:43.060652 31408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:31:43.060846 31408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "a467a0401fb24d69bfa3a101a3e66e2a"
format_stamp: "Formatted at 2026-09-25 09:31:43 on dist-test-slave-9pj8"
server_key: "af60ba78a7c3503f740fbd6ad657b082"
server_key_iv: "118d765458ec9be26875d68774bbbe09"
server_key_version: "encryptionkey@0"
I20260925 09:31:43.061033 31408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:43.081156 31408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:43.084969 31408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:43.085244 31408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:43.103574 31408 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46859
I20260925 09:31:43.103657 31474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46859 every 8 connection(s)
I20260925 09:31:43.104918 31408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:31:43.108800 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:43.111063 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 31408
I20260925 09:31:43.111248 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:31:43.111843 10697 external_mini_cluster.cc:1428] Setting key 854a90528de97a155e259740fc7d9aa8
I20260925 09:31:43.121826 31475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a: Bootstrap starting.
I20260925 09:31:43.125964 31475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a: Neither blocks nor log segments found. Creating new log.
Sep 25 09:31:43 dist-test-slave-9pj8 krb5kdc[31395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 09:31:43.127202 31475 log.cc:824] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:43.130218 31475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a: No bootstrap required, opened a new log
I20260925 09:31:43.135978 31478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:43.113666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46879 (local address 127.0.0.1:46859)
0925 09:31:43.114567 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:43.114578 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:43.114626 (+ 48us) server_negotiation.cc:407] Connection header received
0925 09:31:43.116446 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:43.116468 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:43.116836 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:43.117153 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:43.118992 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:43.120501 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:43.122385 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:43.123099 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:43.123270 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:43.128402 (+ 5132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:43.128410 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:43.128424 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:43.128460 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:43.131546 (+ 3086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:43.132394 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:43.132402 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:43.132407 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:43.132592 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:43.133023 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:43.133028 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:43.133031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:43.133587 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:43.133917 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:43.134441 (+ 524us) server_negotiation.cc:299] Negotiation successful
0925 09:31:43.134866 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":430,"threads_started":1}
I20260925 09:31:43.139446 31475 raft_consensus.cc:348] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a467a0401fb24d69bfa3a101a3e66e2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46859 } }
I20260925 09:31:43.139886 31475 raft_consensus.cc:374] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:43.140002 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:31:43.139981 31475 raft_consensus.cc:729] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a467a0401fb24d69bfa3a101a3e66e2a, State: Initialized, Role: FOLLOWER
I20260925 09:31:43.140522 31475 consensus_queue.cc:260] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a467a0401fb24d69bfa3a101a3e66e2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46859 } }
I20260925 09:31:43.140739 31475 raft_consensus.cc:388] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:31:43.140910 31475 raft_consensus.cc:482] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:31:43.141091 31475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:43.143265 31475 raft_consensus.cc:504] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a467a0401fb24d69bfa3a101a3e66e2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46859 } }
I20260925 09:31:43.143759 31475 leader_election.cc:302] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a467a0401fb24d69bfa3a101a3e66e2a; no voters:
I20260925 09:31:43.144461 31475 leader_election.cc:288] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:31:43.144575 31484 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:43.146754 31475 sys_catalog.cc:564] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:31:43.147006 31484 raft_consensus.cc:686] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a [term 1 LEADER]: Becoming Leader. State: Replica: a467a0401fb24d69bfa3a101a3e66e2a, State: Running, Role: LEADER
I20260925 09:31:43.147576 31484 consensus_queue.cc:237] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a467a0401fb24d69bfa3a101a3e66e2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46859 } }
I20260925 09:31:43.151160 31487 sys_catalog.cc:455] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a467a0401fb24d69bfa3a101a3e66e2a. Latest consensus state: current_term: 1 leader_uuid: "a467a0401fb24d69bfa3a101a3e66e2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a467a0401fb24d69bfa3a101a3e66e2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46859 } } }
I20260925 09:31:43.151546 31487 sys_catalog.cc:458] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:43.152175 31485 sys_catalog.cc:455] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a467a0401fb24d69bfa3a101a3e66e2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a467a0401fb24d69bfa3a101a3e66e2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46859 } } }
I20260925 09:31:43.152555 31485 sys_catalog.cc:458] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:43.155690 31495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:31:43.161675 31495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:31:43.166375 31478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:43.144714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50039 (local address 127.0.0.1:46859)
0925 09:31:43.144898 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:43.144907 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:43.145749 (+ 842us) server_negotiation.cc:407] Connection header received
0925 09:31:43.145992 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:43.146005 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:43.146072 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:43.146182 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:43.147837 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:43.149203 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:43.151409 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:43.152483 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:43.152651 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:43.156781 (+ 4130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:43.156789 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:43.156794 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:43.156837 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:43.161221 (+ 4384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:43.161957 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:43.161965 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:43.161972 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:43.162126 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:43.162694 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:43.162702 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:43.162708 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:43.163303 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:43.165315 (+ 2012us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:43.165519 (+ 204us) server_negotiation.cc:299] Negotiation successful
0925 09:31:43.165850 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 09:31:43.171151 31495 catalog_manager.cc:1347] Generated new cluster ID: a24b442117844c5db0693725a8fc9512
I20260925 09:31:43.171262 31495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:31:43.189837 31495 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:31:43.191130 31495 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:31:43.199851 31495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a467a0401fb24d69bfa3a101a3e66e2a: Generated new TSK 0
I20260925 09:31:43.200629 31495 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:31:43.207890 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 31408
I20260925 09:31:43.209249 31470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:43.296744 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 31408
2026-09-25T09:31:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1970 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:31:43 dist-test-slave-9pj8 krb5kdc[31507](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:31:43 dist-test-slave-9pj8 krb5kdc[31507](info): set up 2 sockets
Sep 25 09:31:43 dist-test-slave-9pj8 krb5kdc[31507](info): commencing operation
krb5kdc: starting...
W20260925 09:31:44.368227 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.007s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:31:44 dist-test-slave-9pj8 krb5kdc[31507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:31:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:31:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:31:44.607086 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39581
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39581
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:44.924713 31520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:44.925060 31520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:44.925231 31520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:44.932161 31520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:31:44.932323 31520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:44.932430 31520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:44.932540 31520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:31:44.932624 31520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:31:44.942351 31520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:44.943722 31520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:44.945062 31520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:44.956933 31528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:44.956930 31526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:44.957193 31525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:44.957841 31520 server_base.cc:1034] running on GCE node
I20260925 09:31:44.958678 31520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:44.960106 31520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:44.961323 31520 hybrid_clock.cc:648] HybridClock initialized: now 1790328704961271 us; error 38 us; skew 500 ppm
I20260925 09:31:44.961692 31520 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:44 dist-test-slave-9pj8 krb5kdc[31507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328704, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:44.968472 31520 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:31:44.969895 31520 webserver.cc:460] Webserver started at http://127.0.0.1:45189/ using document root <none> and password file <none>
I20260925 09:31:44.970494 31520 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:44.970589 31520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:44.970844 31520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:44.973747 31520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "44972265a709415ab1d743273888f023"
format_stamp: "Formatted at 2026-09-25 09:31:44 on dist-test-slave-9pj8"
server_key: "8462b1b385e9a8aca6196ab995a258ea"
server_key_iv: "3820249453f6516d5e3a8d9b88736960"
server_key_version: "encryptionkey@0"
I20260925 09:31:44.974445 31520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "44972265a709415ab1d743273888f023"
format_stamp: "Formatted at 2026-09-25 09:31:44 on dist-test-slave-9pj8"
server_key: "8462b1b385e9a8aca6196ab995a258ea"
server_key_iv: "3820249453f6516d5e3a8d9b88736960"
server_key_version: "encryptionkey@0"
I20260925 09:31:44.979182 31520 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 09:31:44.982322 31535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:44.983513 31520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 09:31:44.983680 31520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "44972265a709415ab1d743273888f023"
format_stamp: "Formatted at 2026-09-25 09:31:44 on dist-test-slave-9pj8"
server_key: "8462b1b385e9a8aca6196ab995a258ea"
server_key_iv: "3820249453f6516d5e3a8d9b88736960"
server_key_version: "encryptionkey@0"
I20260925 09:31:44.983928 31520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:45.012858 31520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:45.017210 31520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:45.017501 31520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:45.035920 31520 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39581
I20260925 09:31:45.035923 31586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39581 every 8 connection(s)
I20260925 09:31:45.037358 31520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:31:45.038950 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 31520
I20260925 09:31:45.039139 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:31:45.039594 10697 external_mini_cluster.cc:1428] Setting key ae489b99afc382868c334093bf8872c0
I20260925 09:31:45.041584 31587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:45.054178 31590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:45.041464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46273 (local address 127.0.0.1:39581)
0925 09:31:45.042265 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:45.042289 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:45.042336 (+ 47us) server_negotiation.cc:407] Connection header received
0925 09:31:45.044469 (+ 2133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:45.044495 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:45.044987 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:45.045420 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:45.045900 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:45.047851 (+ 1951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:45.049113 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:45.050158 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:45.050372 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:45.051456 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:45.051465 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:45.051484 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:45.052232 (+ 748us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:45.052333 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:45.052362 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:45.052487 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:45.052673 (+ 186us) server_negotiation.cc:299] Negotiation successful
0925 09:31:45.052945 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":358,"threads_started":1}
I20260925 09:31:45.055501 31587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023: Bootstrap starting.
I20260925 09:31:45.057636 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:31:45.058794 31587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:45.059935 31587 log.cc:824] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:45.062398 31587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023: No bootstrap required, opened a new log
I20260925 09:31:45.069093 31590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:45.062792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50775 (local address 127.0.0.1:39581)
0925 09:31:45.063024 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:45.063030 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:45.063338 (+ 308us) server_negotiation.cc:407] Connection header received
0925 09:31:45.063506 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:45.063510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:45.063562 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:45.063635 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:45.064225 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:45.064977 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:45.066822 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:45.067277 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:45.067391 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:45.068418 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:45.068422 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:45.068425 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:45.068439 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:45.068477 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:45.068491 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:45.068561 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:45.068716 (+ 155us) server_negotiation.cc:299] Negotiation successful
0925 09:31:45.068772 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 09:31:45.071739 31587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44972265a709415ab1d743273888f023" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39581 } }
I20260925 09:31:45.072139 31587 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:45.072254 31587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44972265a709415ab1d743273888f023, State: Initialized, Role: FOLLOWER
I20260925 09:31:45.072764 31587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44972265a709415ab1d743273888f023" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39581 } }
I20260925 09:31:45.072988 31587 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:31:45.073128 31587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:31:45.073295 31587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:45.075270 31587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44972265a709415ab1d743273888f023" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39581 } }
I20260925 09:31:45.075728 31587 leader_election.cc:302] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44972265a709415ab1d743273888f023; no voters:
I20260925 09:31:45.076325 31587 leader_election.cc:288] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:31:45.076515 31597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:45.077565 31597 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023 [term 1 LEADER]: Becoming Leader. State: Replica: 44972265a709415ab1d743273888f023, State: Running, Role: LEADER
I20260925 09:31:45.078089 31587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:31:45.078004 31597 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44972265a709415ab1d743273888f023" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39581 } }
I20260925 09:31:45.081143 31598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44972265a709415ab1d743273888f023" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44972265a709415ab1d743273888f023" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39581 } } }
I20260925 09:31:45.081132 31599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44972265a709415ab1d743273888f023. Latest consensus state: current_term: 1 leader_uuid: "44972265a709415ab1d743273888f023" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44972265a709415ab1d743273888f023" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39581 } } }
I20260925 09:31:45.081634 31598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:45.081652 31599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:45.083016 31603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:31:45.088366 31603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:31:45.096643 31603 catalog_manager.cc:1347] Generated new cluster ID: 24971ba978894daa82a4f445cddf50de
I20260925 09:31:45.096745 31603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:31:45.111678 31603 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:31:45.113359 31603 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:31:45.128883 31603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44972265a709415ab1d743273888f023: Generated new TSK 0
I20260925 09:31:45.129699 31603 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:31:45.147166 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 31520
W20260925 09:31:45.148336 31582 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46273
W20260925 09:31:45.149046 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 31520: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:31:45.149199 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 31520
2026-09-25T09:31:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1851 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:31:45 dist-test-slave-9pj8 krb5kdc[31618](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:31:45 dist-test-slave-9pj8 krb5kdc[31618](info): set up 2 sockets
Sep 25 09:31:45 dist-test-slave-9pj8 krb5kdc[31618](info): commencing operation
krb5kdc: starting...
W20260925 09:31:46.228384 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:31:46 dist-test-slave-9pj8 krb5kdc[31618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:31:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:31:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:31:46.454700 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36209
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:46.771085 31631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:46.771344 31631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:46.771503 31631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:46.778394 31631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:31:46.778497 31631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:46.778577 31631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:46.778677 31631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:31:46.778752 31631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:31:46.788651 31631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:46.789824 31631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:46.791112 31631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:46.802523 31637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:46.802561 31636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:46.802714 31639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:46.803383 31631 server_base.cc:1034] running on GCE node
I20260925 09:31:46.804183 31631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:46.805563 31631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:46.806852 31631 hybrid_clock.cc:648] HybridClock initialized: now 1790328706806795 us; error 45 us; skew 500 ppm
I20260925 09:31:46.807199 31631 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:46 dist-test-slave-9pj8 krb5kdc[31618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328706, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:46.810319 31631 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:31:46.811751 31631 webserver.cc:460] Webserver started at http://127.0.0.1:45913/ using document root <none> and password file <none>
I20260925 09:31:46.812462 31631 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:46.812615 31631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:46.812916 31631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:46.815698 31631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "36752b17669940e4ac6c06a9a61d3b89"
format_stamp: "Formatted at 2026-09-25 09:31:46 on dist-test-slave-9pj8"
server_key: "d0ef3e07130021c61089a205898eed61"
server_key_iv: "3b63d18863d5ed815b673213d9dd63a0"
server_key_version: "encryptionkey@0"
I20260925 09:31:46.816524 31631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "36752b17669940e4ac6c06a9a61d3b89"
format_stamp: "Formatted at 2026-09-25 09:31:46 on dist-test-slave-9pj8"
server_key: "d0ef3e07130021c61089a205898eed61"
server_key_iv: "3b63d18863d5ed815b673213d9dd63a0"
server_key_version: "encryptionkey@0"
I20260925 09:31:46.821462 31631 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 09:31:46.825172 31646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:46.826833 31631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 09:31:46.827025 31631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "36752b17669940e4ac6c06a9a61d3b89"
format_stamp: "Formatted at 2026-09-25 09:31:46 on dist-test-slave-9pj8"
server_key: "d0ef3e07130021c61089a205898eed61"
server_key_iv: "3b63d18863d5ed815b673213d9dd63a0"
server_key_version: "encryptionkey@0"
I20260925 09:31:46.827196 31631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:46.832675 31631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:46.836440 31631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:46.836700 31631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:46.854148 31631 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36209
I20260925 09:31:46.854363 31697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36209 every 8 connection(s)
I20260925 09:31:46.855443 31631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:31:46.859392 31698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:46.864717 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 31631
I20260925 09:31:46.864907 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:31:46.865543 10697 external_mini_cluster.cc:1428] Setting key fac5142d392a0bec3aa3882fa3a4c74b
I20260925 09:31:46.871829 31698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89: Bootstrap starting.
I20260925 09:31:46.873512 31701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:46.867399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41695 (local address 127.0.0.1:36209)
0925 09:31:46.868168 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:46.868191 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:46.868228 (+ 37us) server_negotiation.cc:407] Connection header received
0925 09:31:46.870316 (+ 2088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:46.870335 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:46.870658 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:46.870954 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:46.872022 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:46.872028 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:46.872040 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:46.872054 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:46.872116 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:46.872135 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:46.872296 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:46.872450 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 09:31:46.872650 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":368,"threads_started":1}
I20260925 09:31:46.874903 31698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:46.876086 31698 log.cc:824] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:46.877166 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:31:46.879097 31698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89: No bootstrap required, opened a new log
I20260925 09:31:46.884907 31701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:46.882332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45173 (local address 127.0.0.1:36209)
0925 09:31:46.882512 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:46.882518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:46.882882 (+ 364us) server_negotiation.cc:407] Connection header received
0925 09:31:46.883014 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:46.883024 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:46.883077 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:46.883195 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:46.884168 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:46.884173 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:46.884178 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:46.884195 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:46.884249 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:46.884268 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:46.884392 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:46.884520 (+ 128us) server_negotiation.cc:299] Negotiation successful
0925 09:31:46.884580 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 09:31:46.888677 31698 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36752b17669940e4ac6c06a9a61d3b89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36209 } }
I20260925 09:31:46.888990 31698 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:46.889057 31698 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36752b17669940e4ac6c06a9a61d3b89, State: Initialized, Role: FOLLOWER
I20260925 09:31:46.889503 31698 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36752b17669940e4ac6c06a9a61d3b89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36209 } }
I20260925 09:31:46.889780 31698 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:31:46.889919 31698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:31:46.890141 31698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:46.892499 31698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36752b17669940e4ac6c06a9a61d3b89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36209 } }
I20260925 09:31:46.892921 31698 leader_election.cc:302] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36752b17669940e4ac6c06a9a61d3b89; no voters:
I20260925 09:31:46.893407 31698 leader_election.cc:288] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:31:46.893558 31708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:46.894196 31708 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89 [term 1 LEADER]: Becoming Leader. State: Replica: 36752b17669940e4ac6c06a9a61d3b89, State: Running, Role: LEADER
I20260925 09:31:46.894799 31708 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36752b17669940e4ac6c06a9a61d3b89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36209 } }
I20260925 09:31:46.895380 31698 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:31:46.897764 31710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36752b17669940e4ac6c06a9a61d3b89. Latest consensus state: current_term: 1 leader_uuid: "36752b17669940e4ac6c06a9a61d3b89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36752b17669940e4ac6c06a9a61d3b89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36209 } } }
I20260925 09:31:46.898110 31710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:46.899747 31714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:31:46.899540 31709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36752b17669940e4ac6c06a9a61d3b89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36752b17669940e4ac6c06a9a61d3b89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36209 } } }
I20260925 09:31:46.900024 31709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:46.907228 31714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:31:46.916620 31714 catalog_manager.cc:1347] Generated new cluster ID: 53044746b1124c7884c3275f41dadbc6
I20260925 09:31:46.916756 31714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:31:46.935029 31714 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:31:46.935503 31714 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:31:46.947113 31714 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36752b17669940e4ac6c06a9a61d3b89: Generated new TSK 0
I20260925 09:31:46.948132 31714 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:31:46.957748 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 31631
W20260925 09:31:46.959054 31693 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41695
W20260925 09:31:46.959831 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 31631: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:31:46.960006 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 31631
2026-09-25T09:31:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1815 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:31:47 dist-test-slave-9pj8 krb5kdc[31729](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:31:47 dist-test-slave-9pj8 krb5kdc[31729](info): set up 2 sockets
Sep 25 09:31:47 dist-test-slave-9pj8 krb5kdc[31729](info): commencing operation
krb5kdc: starting...
W20260925 09:31:48.064392 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.006s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:31:48 dist-test-slave-9pj8 krb5kdc[31729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:31:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:31:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:31:48.302740 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43597
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:48.629711 31742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:48.630023 31742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:48.630116 31742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:48.637425 31742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:31:48.637537 31742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:48.637580 31742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:48.637638 31742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:31:48.637679 31742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:31:48.647393 31742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:48.648783 31742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:48.650110 31742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:48.661808 31747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:48.661736 31748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:48.661774 31750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:48.662878 31742 server_base.cc:1034] running on GCE node
I20260925 09:31:48.663592 31742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:48.664929 31742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:48.666157 31742 hybrid_clock.cc:648] HybridClock initialized: now 1790328708666119 us; error 48 us; skew 500 ppm
I20260925 09:31:48.666495 31742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:48 dist-test-slave-9pj8 krb5kdc[31729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328708, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:48.669693 31742 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:31:48.671048 31742 webserver.cc:460] Webserver started at http://127.0.0.1:40427/ using document root <none> and password file <none>
I20260925 09:31:48.671659 31742 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:48.671752 31742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:48.672055 31742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:48.674723 31742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "4f6dc66d6d5641e1a1f318aa239b9d93"
format_stamp: "Formatted at 2026-09-25 09:31:48 on dist-test-slave-9pj8"
server_key: "33445fabd8ef9375aee32e251bac2575"
server_key_iv: "a8aab0316085369f709f00c5f95f9770"
server_key_version: "encryptionkey@0"
I20260925 09:31:48.675413 31742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "4f6dc66d6d5641e1a1f318aa239b9d93"
format_stamp: "Formatted at 2026-09-25 09:31:48 on dist-test-slave-9pj8"
server_key: "33445fabd8ef9375aee32e251bac2575"
server_key_iv: "a8aab0316085369f709f00c5f95f9770"
server_key_version: "encryptionkey@0"
I20260925 09:31:48.680243 31742 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260925 09:31:48.683480 31757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:48.684768 31742 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:31:48.684960 31742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "4f6dc66d6d5641e1a1f318aa239b9d93"
format_stamp: "Formatted at 2026-09-25 09:31:48 on dist-test-slave-9pj8"
server_key: "33445fabd8ef9375aee32e251bac2575"
server_key_iv: "a8aab0316085369f709f00c5f95f9770"
server_key_version: "encryptionkey@0"
I20260925 09:31:48.685137 31742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:48.709961 31742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:48.714421 31742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:48.714711 31742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:48.732906 31742 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43597
I20260925 09:31:48.732954 31808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43597 every 8 connection(s)
I20260925 09:31:48.734263 31742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:31:48.736960 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 31742
I20260925 09:31:48.737130 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:31:48.737560 10697 external_mini_cluster.cc:1428] Setting key 196e7581f2c5b95f84c9040f31860f5f
I20260925 09:31:48.738174 31809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 25 09:31:48 dist-test-slave-9pj8 krb5kdc[31729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 09:31:48.751993 31809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93: Bootstrap starting.
I20260925 09:31:48.755956 31809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:48.757274 31809 log.cc:824] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:48.760789 31809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93: No bootstrap required, opened a new log
I20260925 09:31:48.761721 31812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:48.739370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:46157 (local address 127.0.0.1:43597)
0925 09:31:48.740139 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:48.740154 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:48.740201 (+ 47us) server_negotiation.cc:407] Connection header received
0925 09:31:48.741857 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:48.741884 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:48.742416 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:48.742871 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:48.744213 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:48.746275 (+ 2062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:48.747649 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:48.748704 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:48.748913 (+ 209us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:48.752918 (+ 4005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:48.752928 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:48.752945 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:48.752990 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:48.756946 (+ 3956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:48.757796 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:48.757808 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:48.757819 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:48.758036 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:48.758542 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:48.758545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:48.758547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:48.759268 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:48.759575 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:48.759990 (+ 415us) server_negotiation.cc:299] Negotiation successful
0925 09:31:48.760342 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":347,"threads_started":1}
I20260925 09:31:48.765874 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:31:48.769233 31809 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6dc66d6d5641e1a1f318aa239b9d93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43597 } }
I20260925 09:31:48.769599 31809 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:48.769701 31809 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f6dc66d6d5641e1a1f318aa239b9d93, State: Initialized, Role: FOLLOWER
I20260925 09:31:48.770262 31809 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6dc66d6d5641e1a1f318aa239b9d93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43597 } }
I20260925 09:31:48.770505 31809 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:31:48.770658 31809 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:31:48.770833 31809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:48.773989 31809 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6dc66d6d5641e1a1f318aa239b9d93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43597 } }
I20260925 09:31:48.774516 31809 leader_election.cc:302] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f6dc66d6d5641e1a1f318aa239b9d93; no voters:
I20260925 09:31:48.775197 31809 leader_election.cc:288] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:31:48.775305 31819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:48.777418 31809 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:31:48.778123 31819 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93 [term 1 LEADER]: Becoming Leader. State: Replica: 4f6dc66d6d5641e1a1f318aa239b9d93, State: Running, Role: LEADER
I20260925 09:31:48.778868 31819 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6dc66d6d5641e1a1f318aa239b9d93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43597 } }
I20260925 09:31:48.782024 31820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f6dc66d6d5641e1a1f318aa239b9d93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6dc66d6d5641e1a1f318aa239b9d93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43597 } } }
I20260925 09:31:48.782467 31820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:48.782724 31821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f6dc66d6d5641e1a1f318aa239b9d93. Latest consensus state: current_term: 1 leader_uuid: "4f6dc66d6d5641e1a1f318aa239b9d93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6dc66d6d5641e1a1f318aa239b9d93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43597 } } }
I20260925 09:31:48.783071 31821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:48.786872 31829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:31:48.789757 31812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:48.770570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:46169 (local address 127.0.0.1:43597)
0925 09:31:48.770751 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:48.770757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:48.771397 (+ 640us) server_negotiation.cc:407] Connection header received
0925 09:31:48.771553 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:48.771561 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:48.771622 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:48.771716 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:48.773492 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:48.774371 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:48.776256 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:48.776986 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:48.777076 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:48.780957 (+ 3881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:48.780966 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:48.780974 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:48.781025 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:48.785987 (+ 4962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:48.787026 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:48.787035 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:48.787041 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:48.787294 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:48.787718 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:48.787723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:48.787725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:48.788183 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:48.788493 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:48.789042 (+ 549us) server_negotiation.cc:299] Negotiation successful
0925 09:31:48.789243 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 09:31:48.792979 31829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:31:48.801424 31829 catalog_manager.cc:1347] Generated new cluster ID: 14a5cdbbe30d4562a969dec68b68f352
I20260925 09:31:48.801527 31829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:31:48.822075 31829 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:31:48.823288 31829 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:31:48.835137 31829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f6dc66d6d5641e1a1f318aa239b9d93: Generated new TSK 0
I20260925 09:31:48.836078 31829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:31:48.841387 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 31742
I20260925 09:31:48.842626 31804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:48.932803 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 31742
W20260925 09:31:48.949935 31727 connection.cc:535] client connection to 127.0.0.1:43597 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43597): Connection reset by peer (error 104)
2026-09-25T09:31:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1970 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:31:49 dist-test-slave-9pj8 krb5kdc[31841](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:31:49 dist-test-slave-9pj8 krb5kdc[31841](info): set up 2 sockets
Sep 25 09:31:49 dist-test-slave-9pj8 krb5kdc[31841](info): commencing operation
krb5kdc: starting...
W20260925 09:31:50.016485 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.001s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:31:50 dist-test-slave-9pj8 krb5kdc[31841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:31:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:31:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:31:50.259346 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34155
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:50.577298 31854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:50.577560 31854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:50.577718 31854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:50.584674 31854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:31:50.584770 31854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:50.584864 31854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:50.584964 31854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:31:50.585044 31854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:31:50.594004 31854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:50.595233 31854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:50.596547 31854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:50.607649 31862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:50.607621 31860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:50.607708 31859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:50.608459 31854 server_base.cc:1034] running on GCE node
I20260925 09:31:50.609261 31854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:50.610680 31854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:50.611876 31854 hybrid_clock.cc:648] HybridClock initialized: now 1790328710611821 us; error 49 us; skew 500 ppm
I20260925 09:31:50.612212 31854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:50 dist-test-slave-9pj8 krb5kdc[31841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328710, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:50.618528 31854 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:31:50.619987 31854 webserver.cc:460] Webserver started at http://127.0.0.1:40363/ using document root <none> and password file <none>
I20260925 09:31:50.620711 31854 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:50.620880 31854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:50.621169 31854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:50.624029 31854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "f0f6473757634b5980f0311135769089"
format_stamp: "Formatted at 2026-09-25 09:31:50 on dist-test-slave-9pj8"
server_key: "923f1afa77b3ebabec90c0b3d3d692dd"
server_key_iv: "f37dad9dcbb0098e547b9a3bcb74a5aa"
server_key_version: "encryptionkey@0"
I20260925 09:31:50.624773 31854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "f0f6473757634b5980f0311135769089"
format_stamp: "Formatted at 2026-09-25 09:31:50 on dist-test-slave-9pj8"
server_key: "923f1afa77b3ebabec90c0b3d3d692dd"
server_key_iv: "f37dad9dcbb0098e547b9a3bcb74a5aa"
server_key_version: "encryptionkey@0"
I20260925 09:31:50.629487 31854 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260925 09:31:50.632752 31869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:50.633983 31854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:31:50.634155 31854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "f0f6473757634b5980f0311135769089"
format_stamp: "Formatted at 2026-09-25 09:31:50 on dist-test-slave-9pj8"
server_key: "923f1afa77b3ebabec90c0b3d3d692dd"
server_key_iv: "f37dad9dcbb0098e547b9a3bcb74a5aa"
server_key_version: "encryptionkey@0"
I20260925 09:31:50.634356 31854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:50.653842 31854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:50.657567 31854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:50.657840 31854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:50.676098 31854 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34155
I20260925 09:31:50.676119 31920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34155 every 8 connection(s)
I20260925 09:31:50.677438 31854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:31:50.681176 31921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:50.681739 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 31854
I20260925 09:31:50.681921 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:31:50.682471 10697 external_mini_cluster.cc:1428] Setting key b81530d05d99c181c6baea99f9fcb8f7
I20260925 09:31:50.695318 31921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089: Bootstrap starting.
I20260925 09:31:50.696475 31924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:50.684340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:59625 (local address 127.0.0.1:34155)
0925 09:31:50.685237 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:50.685252 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:50.685302 (+ 50us) server_negotiation.cc:407] Connection header received
0925 09:31:50.687395 (+ 2093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:50.687419 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:50.687913 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:50.688253 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:50.688815 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:50.690100 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:50.691935 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:50.692659 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:50.692796 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:50.694064 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:50.694071 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:50.694085 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:50.694639 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:50.694719 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:50.694745 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:50.694876 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:50.695170 (+ 294us) server_negotiation.cc:299] Negotiation successful
0925 09:31:50.695439 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":494,"threads_started":1}
I20260925 09:31:50.698576 31921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:50.699808 31921 log.cc:824] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:50.700443 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:31:50.702498 31921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089: No bootstrap required, opened a new log
I20260925 09:31:50.711287 31924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:50.704686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:45199 (local address 127.0.0.1:34155)
0925 09:31:50.704887 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:50.704893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:50.705401 (+ 508us) server_negotiation.cc:407] Connection header received
0925 09:31:50.705547 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:50.705553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:50.705632 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:50.705747 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:50.706207 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:50.707303 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:50.708609 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:50.709311 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:50.709463 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:50.710535 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:50.710541 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:50.710546 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:50.710567 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:50.710622 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:50.710642 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:50.710758 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:50.710928 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 09:31:50.710978 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 09:31:50.712037 31921 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f6473757634b5980f0311135769089" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34155 } }
I20260925 09:31:50.712446 31921 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:50.712560 31921 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0f6473757634b5980f0311135769089, State: Initialized, Role: FOLLOWER
I20260925 09:31:50.713109 31921 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f6473757634b5980f0311135769089" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34155 } }
I20260925 09:31:50.713363 31921 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:31:50.713506 31921 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:31:50.713682 31921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:50.715905 31921 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f6473757634b5980f0311135769089" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34155 } }
I20260925 09:31:50.716387 31921 leader_election.cc:302] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0f6473757634b5980f0311135769089; no voters:
I20260925 09:31:50.717023 31921 leader_election.cc:288] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:31:50.717137 31931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:50.717800 31931 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089 [term 1 LEADER]: Becoming Leader. State: Replica: f0f6473757634b5980f0311135769089, State: Running, Role: LEADER
I20260925 09:31:50.718413 31931 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f6473757634b5980f0311135769089" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34155 } }
I20260925 09:31:50.719379 31921 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:31:50.721257 31933 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0f6473757634b5980f0311135769089. Latest consensus state: current_term: 1 leader_uuid: "f0f6473757634b5980f0311135769089" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f6473757634b5980f0311135769089" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34155 } } }
I20260925 09:31:50.721382 31932 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0f6473757634b5980f0311135769089" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f6473757634b5980f0311135769089" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34155 } } }
I20260925 09:31:50.721731 31933 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:50.721797 31932 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:50.723537 31936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:31:50.728706 31936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:31:50.738476 31936 catalog_manager.cc:1347] Generated new cluster ID: cf9bb14ca2404a06b7cc7159dda3a22a
I20260925 09:31:50.738583 31936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:31:50.750967 31936 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:31:50.752164 31936 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:31:50.760959 31936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0f6473757634b5980f0311135769089: Generated new TSK 0
I20260925 09:31:50.761910 31936 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:31:50.779011 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 31854
W20260925 09:31:50.780646 31916 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.121:59625
W20260925 09:31:50.781468 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 31854: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:31:50.781615 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 31854
W20260925 09:31:50.798785 31839 connection.cc:535] client connection to 127.0.0.1:34155 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34155): Connection reset by peer (error 104)
2026-09-25T09:31:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1848 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:31:50 dist-test-slave-9pj8 krb5kdc[31952](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:31:50 dist-test-slave-9pj8 krb5kdc[31952](info): set up 2 sockets
Sep 25 09:31:50 dist-test-slave-9pj8 krb5kdc[31952](info): commencing operation
krb5kdc: starting...
W20260925 09:31:51.860258 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:31:52 dist-test-slave-9pj8 krb5kdc[31952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:31:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:31:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:31:52.126971 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42821
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:52.450181 31965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:52.450562 31965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:52.450735 31965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:52.458043 31965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:31:52.458191 31965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:52.458292 31965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:52.458396 31965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:31:52.458477 31965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:31:52.467978 31965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:52.469302 31965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:52.470578 31965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:52.481894 31970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:52.481946 31973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:52.482064 31971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:52.482436 31965 server_base.cc:1034] running on GCE node
I20260925 09:31:52.483227 31965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:52.484553 31965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:52.485754 31965 hybrid_clock.cc:648] HybridClock initialized: now 1790328712485706 us; error 46 us; skew 500 ppm
I20260925 09:31:52.486081 31965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:52 dist-test-slave-9pj8 krb5kdc[31952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328712, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:52.489159 31965 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:31:52.490481 31965 webserver.cc:460] Webserver started at http://127.0.0.1:42437/ using document root <none> and password file <none>
I20260925 09:31:52.491119 31965 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:52.491240 31965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:52.491539 31965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:52.494336 31965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "63a04b114d01463dba745584fa7894cb"
format_stamp: "Formatted at 2026-09-25 09:31:52 on dist-test-slave-9pj8"
server_key: "e0e2befa2fbce1c66caa94c07d4538b7"
server_key_iv: "239812f6f06303273fe55e516200b038"
server_key_version: "encryptionkey@0"
I20260925 09:31:52.495071 31965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "63a04b114d01463dba745584fa7894cb"
format_stamp: "Formatted at 2026-09-25 09:31:52 on dist-test-slave-9pj8"
server_key: "e0e2befa2fbce1c66caa94c07d4538b7"
server_key_iv: "239812f6f06303273fe55e516200b038"
server_key_version: "encryptionkey@0"
I20260925 09:31:52.501195 31965 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260925 09:31:52.505880 31980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:52.507422 31965 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260925 09:31:52.507596 31965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "63a04b114d01463dba745584fa7894cb"
format_stamp: "Formatted at 2026-09-25 09:31:52 on dist-test-slave-9pj8"
server_key: "e0e2befa2fbce1c66caa94c07d4538b7"
server_key_iv: "239812f6f06303273fe55e516200b038"
server_key_version: "encryptionkey@0"
I20260925 09:31:52.507824 31965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:52.513299 31965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:52.517079 31965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:52.517364 31965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:52.535530 31965 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42821
I20260925 09:31:52.535578 32031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42821 every 8 connection(s)
I20260925 09:31:52.536962 31965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:31:52.539763 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 31965
I20260925 09:31:52.539966 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:31:52.540508 10697 external_mini_cluster.cc:1428] Setting key cac894d00596cbec4680beea576f129d
I20260925 09:31:52.541109 32032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:52.549279 32035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:52.542438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:40359 (local address 127.0.0.1:42821)
0925 09:31:52.543204 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:52.543215 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:52.543255 (+ 40us) server_negotiation.cc:407] Connection header received
0925 09:31:52.545798 (+ 2543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:52.545819 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:52.546149 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:52.546606 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:52.547635 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:52.547640 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:52.547653 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:52.547673 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:52.547726 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:52.547744 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:52.548069 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:52.548202 (+ 133us) server_negotiation.cc:299] Negotiation successful
0925 09:31:52.548393 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":358,"threads_started":1}
I20260925 09:31:52.552773 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:31:52.553997 32032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb: Bootstrap starting.
I20260925 09:31:52.557333 32032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:52.558408 32032 log.cc:824] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:52.559818 32035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:52.557207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:52595 (local address 127.0.0.1:42821)
0925 09:31:52.557447 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:52.557455 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:52.557684 (+ 229us) server_negotiation.cc:407] Connection header received
0925 09:31:52.557907 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:52.557913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:52.557975 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:52.558083 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:52.559011 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:52.559017 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:52.559022 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:52.559059 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:52.559109 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:52.559138 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:52.559274 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:52.559400 (+ 126us) server_negotiation.cc:299] Negotiation successful
0925 09:31:52.559471 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260925 09:31:52.561602 32032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb: No bootstrap required, opened a new log
I20260925 09:31:52.569098 32032 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a04b114d01463dba745584fa7894cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42821 } }
I20260925 09:31:52.569469 32032 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:52.569547 32032 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63a04b114d01463dba745584fa7894cb, State: Initialized, Role: FOLLOWER
I20260925 09:31:52.570092 32032 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a04b114d01463dba745584fa7894cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42821 } }
I20260925 09:31:52.570328 32032 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:31:52.570487 32032 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:31:52.570674 32032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:52.573192 32032 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a04b114d01463dba745584fa7894cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42821 } }
I20260925 09:31:52.573698 32032 leader_election.cc:302] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63a04b114d01463dba745584fa7894cb; no voters:
I20260925 09:31:52.574334 32032 leader_election.cc:288] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:31:52.574476 32042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:52.575721 32042 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb [term 1 LEADER]: Becoming Leader. State: Replica: 63a04b114d01463dba745584fa7894cb, State: Running, Role: LEADER
I20260925 09:31:52.576257 32042 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a04b114d01463dba745584fa7894cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42821 } }
I20260925 09:31:52.576604 32032 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:31:52.578768 32044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63a04b114d01463dba745584fa7894cb. Latest consensus state: current_term: 1 leader_uuid: "63a04b114d01463dba745584fa7894cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a04b114d01463dba745584fa7894cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42821 } } }
I20260925 09:31:52.579419 32044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:52.579643 32043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63a04b114d01463dba745584fa7894cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a04b114d01463dba745584fa7894cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42821 } } }
I20260925 09:31:52.580164 32043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:52.580792 32051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:31:52.586298 32051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:31:52.593869 32051 catalog_manager.cc:1347] Generated new cluster ID: a0376e92eb4e49f2a7fd623e04a68baf
I20260925 09:31:52.593964 32051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:31:52.613391 32051 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:31:52.613790 32051 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:31:52.625371 32051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63a04b114d01463dba745584fa7894cb: Generated new TSK 0
I20260925 09:31:52.626251 32051 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:31:52.641328 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 31965
W20260925 09:31:52.642477 32027 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.121:40359
W20260925 09:31:52.643174 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 31965: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:31:52.643337 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 31965
2026-09-25T09:31:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1861 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:31:52 dist-test-slave-9pj8 krb5kdc[32063](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:31:52 dist-test-slave-9pj8 krb5kdc[32063](info): set up 2 sockets
Sep 25 09:31:52 dist-test-slave-9pj8 krb5kdc[32063](info): commencing operation
krb5kdc: starting...
W20260925 09:31:53.724334 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:31:53 dist-test-slave-9pj8 krb5kdc[32063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:31:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:31:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:31:53.971248 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34799
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:54.294278 32076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:54.294560 32076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:54.294735 32076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:54.302065 32076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:31:54.302175 32076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:54.302255 32076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:54.302337 32076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:31:54.302402 32076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:31:54.312549 32076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34799
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:54.313872 32076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:54.315198 32076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:54.327057 32081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:54.327044 32084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:54.327099 32082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:54.328292 32076 server_base.cc:1034] running on GCE node
I20260925 09:31:54.328992 32076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:54.330322 32076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:54.331542 32076 hybrid_clock.cc:648] HybridClock initialized: now 1790328714331489 us; error 50 us; skew 500 ppm
I20260925 09:31:54.331924 32076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:54 dist-test-slave-9pj8 krb5kdc[32063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328714, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:54.334844 32076 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:31:54.336279 32076 webserver.cc:460] Webserver started at http://127.0.0.1:43855/ using document root <none> and password file <none>
I20260925 09:31:54.336942 32076 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:54.337085 32076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:54.337380 32076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:54.340346 32076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "c2c73d8dfb4346df9142fa17ba7142ff"
format_stamp: "Formatted at 2026-09-25 09:31:54 on dist-test-slave-9pj8"
server_key: "43afe77de9591e5a402db068a9b54d4b"
server_key_iv: "978d4a41118da5dfc27fba0627fd1bd3"
server_key_version: "encryptionkey@0"
I20260925 09:31:54.341089 32076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "c2c73d8dfb4346df9142fa17ba7142ff"
format_stamp: "Formatted at 2026-09-25 09:31:54 on dist-test-slave-9pj8"
server_key: "43afe77de9591e5a402db068a9b54d4b"
server_key_iv: "978d4a41118da5dfc27fba0627fd1bd3"
server_key_version: "encryptionkey@0"
I20260925 09:31:54.345989 32076 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 09:31:54.349236 32091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:54.350459 32076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 09:31:54.350632 32076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "c2c73d8dfb4346df9142fa17ba7142ff"
format_stamp: "Formatted at 2026-09-25 09:31:54 on dist-test-slave-9pj8"
server_key: "43afe77de9591e5a402db068a9b54d4b"
server_key_iv: "978d4a41118da5dfc27fba0627fd1bd3"
server_key_version: "encryptionkey@0"
I20260925 09:31:54.350807 32076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:54.366941 32076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:54.370694 32076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:54.370980 32076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:54.389592 32076 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34799
I20260925 09:31:54.389614 32142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34799 every 8 connection(s)
I20260925 09:31:54.390849 32076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:31:54.393293 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 32076
I20260925 09:31:54.393465 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:31:54.393895 10697 external_mini_cluster.cc:1428] Setting key 6985cd57c37334706a079a42839f6761
I20260925 09:31:54.394462 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 25 09:31:54 dist-test-slave-9pj8 krb5kdc[32063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 09:31:54.408280 32143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff: Bootstrap starting.
I20260925 09:31:54.412197 32143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:54.413409 32143 log.cc:824] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:54.416249 32143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff: No bootstrap required, opened a new log
I20260925 09:31:54.416954 32146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:54.395518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:43743 (local address 127.0.0.1:34799)
0925 09:31:54.396295 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:54.396309 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:54.396356 (+ 47us) server_negotiation.cc:407] Connection header received
0925 09:31:54.398119 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:54.398144 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:54.398631 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:54.399060 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:54.400375 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:54.402391 (+ 2016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:54.403674 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:54.404715 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:54.404912 (+ 197us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:54.409083 (+ 4171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:54.409091 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:54.409108 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:54.409150 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:54.412936 (+ 3786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:54.413624 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:54.413631 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:54.413636 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:54.413809 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:54.414279 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:54.414283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:54.414285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:54.414802 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:54.415119 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:54.415620 (+ 501us) server_negotiation.cc:299] Negotiation successful
0925 09:31:54.415941 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":323,"threads_started":1}
I20260925 09:31:54.420526 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:31:54.424079 32143 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2c73d8dfb4346df9142fa17ba7142ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34799 } }
I20260925 09:31:54.424449 32143 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:54.424551 32143 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2c73d8dfb4346df9142fa17ba7142ff, State: Initialized, Role: FOLLOWER
I20260925 09:31:54.425130 32143 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2c73d8dfb4346df9142fa17ba7142ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34799 } }
I20260925 09:31:54.425361 32143 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:31:54.425503 32143 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:31:54.425686 32143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:54.427702 32143 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2c73d8dfb4346df9142fa17ba7142ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34799 } }
I20260925 09:31:54.428258 32143 leader_election.cc:302] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2c73d8dfb4346df9142fa17ba7142ff; no voters:
I20260925 09:31:54.428901 32143 leader_election.cc:288] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:31:54.429004 32153 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:54.431244 32143 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:31:54.435562 32153 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff [term 1 LEADER]: Becoming Leader. State: Replica: c2c73d8dfb4346df9142fa17ba7142ff, State: Running, Role: LEADER
I20260925 09:31:54.436249 32153 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2c73d8dfb4346df9142fa17ba7142ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34799 } }
I20260925 09:31:54.439425 32154 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2c73d8dfb4346df9142fa17ba7142ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2c73d8dfb4346df9142fa17ba7142ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34799 } } }
I20260925 09:31:54.439968 32154 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:54.440513 32146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:54.424774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:39937 (local address 127.0.0.1:34799)
0925 09:31:54.424969 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:54.424974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:54.425637 (+ 663us) server_negotiation.cc:407] Connection header received
0925 09:31:54.425852 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:54.425858 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:54.425936 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:54.426045 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:54.427449 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:54.428357 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:54.429933 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:54.430500 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:54.430580 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:54.433465 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:54.433471 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:54.433474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:31:54.433502 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:54.436665 (+ 3163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:54.437396 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:54.437401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:54.437405 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:54.437537 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:31:54.438141 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:31:54.438145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:31:54.438148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:31:54.438740 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:54.439067 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:54.439868 (+ 801us) server_negotiation.cc:299] Negotiation successful
0925 09:31:54.440051 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260925 09:31:54.442811 32155 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2c73d8dfb4346df9142fa17ba7142ff. Latest consensus state: current_term: 1 leader_uuid: "c2c73d8dfb4346df9142fa17ba7142ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2c73d8dfb4346df9142fa17ba7142ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34799 } } }
I20260925 09:31:54.443243 32155 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:54.443894 32160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:31:54.449816 32160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:31:54.458773 32160 catalog_manager.cc:1347] Generated new cluster ID: e3d6986773f34fc1ad9245effbc11ab2
I20260925 09:31:54.458895 32160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:31:54.472620 32160 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:31:54.473980 32160 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:31:54.483494 32160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2c73d8dfb4346df9142fa17ba7142ff: Generated new TSK 0
I20260925 09:31:54.484472 32160 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:31:54.490554 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 32076
I20260925 09:31:54.492805 32138 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:31:54.583869 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 32076
W20260925 09:31:54.600925 32061 connection.cc:535] client connection to 127.0.0.1:34799 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34799): Connection reset by peer (error 104)
2026-09-25T09:31:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1939 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:31:54 dist-test-slave-9pj8 krb5kdc[32176](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:31:54 dist-test-slave-9pj8 krb5kdc[32176](info): set up 2 sockets
Sep 25 09:31:54 dist-test-slave-9pj8 krb5kdc[32176](info): commencing operation
krb5kdc: starting...
W20260925 09:31:55.692669 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.012s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:31:55 dist-test-slave-9pj8 krb5kdc[32176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:31:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:31:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:31:55.938908 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38751
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:56.259987 32189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:56.260246 32189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:56.260430 32189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:56.267601 32189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:31:56.267694 32189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:56.267833 32189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:56.267963 32189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:31:56.268036 32189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:31:56.277082 32189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38751
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:56.278291 32189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:56.279553 32189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:56.291200 32194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:56.291159 32195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:56.291236 32197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:56.292047 32189 server_base.cc:1034] running on GCE node
I20260925 09:31:56.292855 32189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:56.294152 32189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:56.295418 32189 hybrid_clock.cc:648] HybridClock initialized: now 1790328716295356 us; error 42 us; skew 500 ppm
I20260925 09:31:56.295861 32189 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:56 dist-test-slave-9pj8 krb5kdc[32176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328716, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:56.302755 32189 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:31:56.304157 32189 webserver.cc:460] Webserver started at http://127.0.0.1:44663/ using document root <none> and password file <none>
I20260925 09:31:56.304773 32189 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:56.304916 32189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:56.305225 32189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:56.308115 32189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "115696a9f54749299fb4b7cc61c26a65"
format_stamp: "Formatted at 2026-09-25 09:31:56 on dist-test-slave-9pj8"
server_key: "dd546bd6cec33e87f476c334b8030fcd"
server_key_iv: "134134a11b6166a089d5e517aa497bdf"
server_key_version: "encryptionkey@0"
I20260925 09:31:56.308844 32189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "115696a9f54749299fb4b7cc61c26a65"
format_stamp: "Formatted at 2026-09-25 09:31:56 on dist-test-slave-9pj8"
server_key: "dd546bd6cec33e87f476c334b8030fcd"
server_key_iv: "134134a11b6166a089d5e517aa497bdf"
server_key_version: "encryptionkey@0"
I20260925 09:31:56.313473 32189 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:31:56.316527 32204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:56.317741 32189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:31:56.317926 32189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "115696a9f54749299fb4b7cc61c26a65"
format_stamp: "Formatted at 2026-09-25 09:31:56 on dist-test-slave-9pj8"
server_key: "dd546bd6cec33e87f476c334b8030fcd"
server_key_iv: "134134a11b6166a089d5e517aa497bdf"
server_key_version: "encryptionkey@0"
I20260925 09:31:56.318102 32189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:56.338299 32189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:56.341908 32189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:56.342159 32189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:56.359982 32189 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38751
I20260925 09:31:56.360020 32255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38751 every 8 connection(s)
I20260925 09:31:56.361334 32189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:31:56.364871 32256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:56.371282 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 32189
I20260925 09:31:56.371474 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:31:56.371939 10697 external_mini_cluster.cc:1428] Setting key f77e41fce4e914adde5ce91e922925e7
I20260925 09:31:56.374778 32256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65: Bootstrap starting.
I20260925 09:31:56.377928 32256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:56.379004 32256 log.cc:824] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:56.381856 32256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65: No bootstrap required, opened a new log
I20260925 09:31:56.384734 32259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:56.373618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:59379 (local address 127.0.0.1:38751)
0925 09:31:56.374338 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:56.374348 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:56.374392 (+ 44us) server_negotiation.cc:407] Connection header received
0925 09:31:56.376112 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:56.376136 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:56.376490 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:56.376770 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:56.377362 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:56.378657 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:56.380411 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:56.381040 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:56.381206 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:56.382529 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:56.382536 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:56.382549 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:56.383018 (+ 469us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:56.383085 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:56.383107 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:56.383213 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:56.383556 (+ 343us) server_negotiation.cc:299] Negotiation successful
0925 09:31:56.383735 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":331,"threads_started":1}
I20260925 09:31:56.388262 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:31:56.389714 32256 raft_consensus.cc:348] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "115696a9f54749299fb4b7cc61c26a65" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38751 } }
I20260925 09:31:56.390051 32256 raft_consensus.cc:374] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:56.390152 32256 raft_consensus.cc:729] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115696a9f54749299fb4b7cc61c26a65, State: Initialized, Role: FOLLOWER
I20260925 09:31:56.390724 32256 consensus_queue.cc:260] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "115696a9f54749299fb4b7cc61c26a65" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38751 } }
I20260925 09:31:56.390975 32256 raft_consensus.cc:388] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:31:56.391125 32256 raft_consensus.cc:482] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:31:56.391291 32256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:56.393802 32256 raft_consensus.cc:504] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "115696a9f54749299fb4b7cc61c26a65" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38751 } }
I20260925 09:31:56.394272 32256 leader_election.cc:302] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 115696a9f54749299fb4b7cc61c26a65; no voters:
I20260925 09:31:56.394963 32256 leader_election.cc:288] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:31:56.395036 32266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:56.397809 32256 sys_catalog.cc:564] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:31:56.398917 32259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:56.392527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:56757 (local address 127.0.0.1:38751)
0925 09:31:56.392693 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:56.392701 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:56.393030 (+ 329us) server_negotiation.cc:407] Connection header received
0925 09:31:56.393200 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:56.393205 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:56.393266 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:56.393362 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:56.393822 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:56.394703 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:56.396484 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:31:56.396934 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:31:56.397054 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:31:56.398159 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:56.398163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:56.398166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:56.398181 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:56.398213 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:56.398226 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:56.398339 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:56.398537 (+ 198us) server_negotiation.cc:299] Negotiation successful
0925 09:31:56.398586 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 09:31:56.401950 32266 raft_consensus.cc:686] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65 [term 1 LEADER]: Becoming Leader. State: Replica: 115696a9f54749299fb4b7cc61c26a65, State: Running, Role: LEADER
I20260925 09:31:56.402609 32266 consensus_queue.cc:237] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "115696a9f54749299fb4b7cc61c26a65" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38751 } }
I20260925 09:31:56.406227 32267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "115696a9f54749299fb4b7cc61c26a65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "115696a9f54749299fb4b7cc61c26a65" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38751 } } }
I20260925 09:31:56.406298 32268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 115696a9f54749299fb4b7cc61c26a65. Latest consensus state: current_term: 1 leader_uuid: "115696a9f54749299fb4b7cc61c26a65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "115696a9f54749299fb4b7cc61c26a65" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38751 } } }
I20260925 09:31:56.406621 32267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:56.406726 32268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:56.408257 32274 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:31:56.413594 32274 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:31:56.422480 32274 catalog_manager.cc:1347] Generated new cluster ID: a64ab1991f014429a0707283d4e41777
I20260925 09:31:56.422607 32274 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:31:56.436717 32274 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:31:56.437896 32274 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:31:56.446187 32274 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 115696a9f54749299fb4b7cc61c26a65: Generated new TSK 0
I20260925 09:31:56.446940 32274 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:31:56.456893 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 32189
W20260925 09:31:56.458607 32251 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.121:59379
W20260925 09:31:56.459410 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 32189: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:31:56.459569 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 32189
W20260925 09:31:56.476331 32174 connection.cc:535] client connection to 127.0.0.1:38751 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38751): Connection reset by peer (error 104)
2026-09-25T09:31:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1875 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:31:56 dist-test-slave-9pj8 krb5kdc[32287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:31:56 dist-test-slave-9pj8 krb5kdc[32287](info): set up 2 sockets
Sep 25 09:31:56 dist-test-slave-9pj8 krb5kdc[32287](info): commencing operation
krb5kdc: starting...
W20260925 09:31:57.548795 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:31:57 dist-test-slave-9pj8 krb5kdc[32287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:31:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:31:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:31:57.784454 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38041
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:58.100605 32300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:58.100878 32300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:58.101058 32300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:58.107939 32300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:31:58.108026 32300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:58.108119 32300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:58.108201 32300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:31:58.108258 32300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:31:58.117595 32300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38041
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:58.118772 32300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:58.120070 32300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:58.132038 32305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:58.131992 32306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:58.131992 32308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:58.132885 32300 server_base.cc:1034] running on GCE node
I20260925 09:31:58.133646 32300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:58.135016 32300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:58.136260 32300 hybrid_clock.cc:648] HybridClock initialized: now 1790328718136197 us; error 45 us; skew 500 ppm
I20260925 09:31:58.136644 32300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:58 dist-test-slave-9pj8 krb5kdc[32287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328718, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:58.139639 32300 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:31:58.141018 32300 webserver.cc:460] Webserver started at http://127.0.0.1:46467/ using document root <none> and password file <none>
I20260925 09:31:58.141655 32300 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:58.141747 32300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:58.141984 32300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:58.144766 32300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "5682c47a8e03487485ae40dd575320b0"
format_stamp: "Formatted at 2026-09-25 09:31:58 on dist-test-slave-9pj8"
server_key: "d121214b6e84c0d8697b3e07d03ee013"
server_key_iv: "c34980d5e3bba411297a553f9762f3dc"
server_key_version: "encryptionkey@0"
I20260925 09:31:58.145522 32300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "5682c47a8e03487485ae40dd575320b0"
format_stamp: "Formatted at 2026-09-25 09:31:58 on dist-test-slave-9pj8"
server_key: "d121214b6e84c0d8697b3e07d03ee013"
server_key_iv: "c34980d5e3bba411297a553f9762f3dc"
server_key_version: "encryptionkey@0"
I20260925 09:31:58.150339 32300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260925 09:31:58.153843 32315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:58.155275 32300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:31:58.155429 32300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "5682c47a8e03487485ae40dd575320b0"
format_stamp: "Formatted at 2026-09-25 09:31:58 on dist-test-slave-9pj8"
server_key: "d121214b6e84c0d8697b3e07d03ee013"
server_key_iv: "c34980d5e3bba411297a553f9762f3dc"
server_key_version: "encryptionkey@0"
I20260925 09:31:58.155601 32300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:31:58.160934 32300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:31:58.164602 32300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:31:58.164826 32300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:31:58.182390 32300 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38041
I20260925 09:31:58.182468 32366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38041 every 8 connection(s)
I20260925 09:31:58.183735 32300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:31:58.186026 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 32300
I20260925 09:31:58.186208 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:31:58.186635 10697 external_mini_cluster.cc:1428] Setting key fb0b0b6144aeeaf24351142dfa14ca39
I20260925 09:31:58.187511 32367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:31:58.193800 32370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:58.188355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:47895 (local address 127.0.0.1:38041)
0925 09:31:58.189080 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:58.189091 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:58.189140 (+ 49us) server_negotiation.cc:407] Connection header received
0925 09:31:58.190520 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:58.190538 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:58.190870 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:58.191266 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:58.192296 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:58.192301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:58.192313 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:58.192339 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:58.192425 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:58.192443 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:58.192687 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:58.192827 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 09:31:58.193004 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":315,"threads_started":1}
I20260925 09:31:58.197254 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:31:58.199287 32367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0: Bootstrap starting.
I20260925 09:31:58.202494 32367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0: Neither blocks nor log segments found. Creating new log.
I20260925 09:31:58.203860 32370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:31:58.201550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:59611 (local address 127.0.0.1:38041)
0925 09:31:58.201740 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 09:31:58.201744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:31:58.202049 (+ 305us) server_negotiation.cc:407] Connection header received
0925 09:31:58.202185 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:31:58.202188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:31:58.202225 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:31:58.202293 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:31:58.203171 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:31:58.203175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:31:58.203178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:31:58.203191 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:31:58.203225 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:31:58.203238 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:31:58.203319 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 09:31:58.203460 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 09:31:58.203502 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 09:31:58.203828 32367 log.cc:824] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0: Log is configured to *not* fsync() on all Append() calls
I20260925 09:31:58.206489 32367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0: No bootstrap required, opened a new log
I20260925 09:31:58.213239 32367 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5682c47a8e03487485ae40dd575320b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38041 } }
I20260925 09:31:58.213564 32367 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:31:58.213671 32367 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5682c47a8e03487485ae40dd575320b0, State: Initialized, Role: FOLLOWER
I20260925 09:31:58.214164 32367 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5682c47a8e03487485ae40dd575320b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38041 } }
I20260925 09:31:58.214367 32367 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:31:58.214512 32367 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:31:58.214694 32367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:31:58.216799 32367 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5682c47a8e03487485ae40dd575320b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38041 } }
I20260925 09:31:58.217252 32367 leader_election.cc:302] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5682c47a8e03487485ae40dd575320b0; no voters:
I20260925 09:31:58.217756 32367 leader_election.cc:288] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:31:58.217913 32377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:31:58.218617 32377 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0 [term 1 LEADER]: Becoming Leader. State: Replica: 5682c47a8e03487485ae40dd575320b0, State: Running, Role: LEADER
I20260925 09:31:58.219120 32377 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5682c47a8e03487485ae40dd575320b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38041 } }
I20260925 09:31:58.219762 32367 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:31:58.222287 32378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5682c47a8e03487485ae40dd575320b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5682c47a8e03487485ae40dd575320b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38041 } } }
I20260925 09:31:58.222299 32379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5682c47a8e03487485ae40dd575320b0. Latest consensus state: current_term: 1 leader_uuid: "5682c47a8e03487485ae40dd575320b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5682c47a8e03487485ae40dd575320b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38041 } } }
I20260925 09:31:58.223112 32378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:58.223165 32379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0 [sys.catalog]: This master's current role is: LEADER
I20260925 09:31:58.224347 32384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:31:58.229017 32384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:31:58.236871 32384 catalog_manager.cc:1347] Generated new cluster ID: 9554e5e2deeb46c8b25ef07f3b9cfcd2
I20260925 09:31:58.236980 32384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:31:58.259554 32384 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:31:58.260056 32384 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:31:58.273247 32384 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5682c47a8e03487485ae40dd575320b0: Generated new TSK 0
I20260925 09:31:58.274204 32384 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:31:58.286597 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 32300
W20260925 09:31:58.287659 32362 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.121:47895
W20260925 09:31:58.288376 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 32300: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:31:58.288496 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 32300
2026-09-25T09:31:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1826 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:31:58 dist-test-slave-9pj8 krb5kdc[32398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:31:58 dist-test-slave-9pj8 krb5kdc[32398](info): set up 2 sockets
Sep 25 09:31:58 dist-test-slave-9pj8 krb5kdc[32398](info): commencing operation
krb5kdc: starting...
W20260925 09:31:59.360414 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:31:59 dist-test-slave-9pj8 krb5kdc[32398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:31:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:31:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:31:59.591104 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:41427
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:41427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:31:59.907100 32411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:31:59.907466 32411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:31:59.907641 32411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:31:59.915047 32411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:31:59.915195 32411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:31:59.915324 32411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:31:59.915443 32411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:31:59.915529 32411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:31:59.925534 32411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:41427
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:41427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:31:59.926877 32411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:31:59.928269 32411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:31:59.939989 32417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:59.939952 32419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:31:59.940625 32416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:31:59.941546 32411 server_base.cc:1034] running on GCE node
I20260925 09:31:59.942445 32411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:31:59.944015 32411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:31:59.945240 32411 hybrid_clock.cc:648] HybridClock initialized: now 1790328719945174 us; error 59 us; skew 500 ppm
I20260925 09:31:59.945621 32411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:31:59 dist-test-slave-9pj8 krb5kdc[32398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328719, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:31:59.949149 32411 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:31:59.950785 32411 webserver.cc:460] Webserver started at http://127.10.114.126:46551/ using document root <none> and password file <none>
I20260925 09:31:59.951473 32411 fs_manager.cc:359] Metadata directory not provided
I20260925 09:31:59.951602 32411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:31:59.951972 32411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:31:59.954937 32411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "dea458ba48c149dfb26462a945579e15"
format_stamp: "Formatted at 2026-09-25 09:31:59 on dist-test-slave-9pj8"
server_key: "82d1a4bb6ccc05febda5950a83256822"
server_key_iv: "4dd2d8ec98bbde6ab71e747de71b81ab"
server_key_version: "encryptionkey@0"
I20260925 09:31:59.955694 32411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "dea458ba48c149dfb26462a945579e15"
format_stamp: "Formatted at 2026-09-25 09:31:59 on dist-test-slave-9pj8"
server_key: "82d1a4bb6ccc05febda5950a83256822"
server_key_iv: "4dd2d8ec98bbde6ab71e747de71b81ab"
server_key_version: "encryptionkey@0"
I20260925 09:31:59.960964 32411 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260925 09:31:59.964628 32426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:31:59.965988 32411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:31:59.966180 32411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "dea458ba48c149dfb26462a945579e15"
format_stamp: "Formatted at 2026-09-25 09:31:59 on dist-test-slave-9pj8"
server_key: "82d1a4bb6ccc05febda5950a83256822"
server_key_iv: "4dd2d8ec98bbde6ab71e747de71b81ab"
server_key_version: "encryptionkey@0"
I20260925 09:31:59.966380 32411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:32:00.004810 32411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:32:00.009572 32411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:32:00.009912 32411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:32:00.028831 32411 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:41427
I20260925 09:32:00.028854 32477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:41427 every 8 connection(s)
I20260925 09:32:00.030256 32411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:32:00.033615 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 32411
I20260925 09:32:00.033799 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:32:00.034039 32478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:32:00.034291 10697 external_mini_cluster.cc:1428] Setting key a8fb8e9146e62fd4978fbf20a90f4208
I20260925 09:32:00.048511 32478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15: Bootstrap starting.
Sep 25 09:32:00 dist-test-slave-9pj8 krb5kdc[32398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:32:00.052435 32478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15: Neither blocks nor log segments found. Creating new log.
I20260925 09:32:00.053785 32478 log.cc:824] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15: Log is configured to *not* fsync() on all Append() calls
I20260925 09:32:00.057258 32478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15: No bootstrap required, opened a new log
I20260925 09:32:00.058943 32481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:00.036165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59593 (local address 127.10.114.126:41427)
0925 09:32:00.036987 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:00.037003 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:00.037053 (+ 50us) server_negotiation.cc:407] Connection header received
0925 09:32:00.039101 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:00.039129 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:00.039685 (+ 556us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:00.040164 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:00.041437 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:00.043424 (+ 1987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:00.044778 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:00.045877 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:00.046066 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:00.050256 (+ 4190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:00.050267 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:00.050287 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:00.050344 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:00.054170 (+ 3826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:00.055189 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:00.055197 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:00.055204 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:00.055404 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:00.055868 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:00.055873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:00.055876 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:00.056562 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:00.056882 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:00.057258 (+ 376us) server_negotiation.cc:299] Negotiation successful
0925 09:32:00.057599 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":335,"threads_started":1}
I20260925 09:32:00.063012 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:32:00.067895 32478 raft_consensus.cc:348] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea458ba48c149dfb26462a945579e15" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 41427 } }
I20260925 09:32:00.068451 32478 raft_consensus.cc:374] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:32:00.068569 32478 raft_consensus.cc:729] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dea458ba48c149dfb26462a945579e15, State: Initialized, Role: FOLLOWER
I20260925 09:32:00.069257 32478 consensus_queue.cc:260] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea458ba48c149dfb26462a945579e15" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 41427 } }
I20260925 09:32:00.069574 32478 raft_consensus.cc:388] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:32:00.069754 32478 raft_consensus.cc:482] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:32:00.069950 32478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:32:00.073047 32478 raft_consensus.cc:504] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea458ba48c149dfb26462a945579e15" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 41427 } }
I20260925 09:32:00.073607 32478 leader_election.cc:302] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dea458ba48c149dfb26462a945579e15; no voters:
I20260925 09:32:00.074321 32478 leader_election.cc:288] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:32:00.074438 32488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:32:00.078954 32488 raft_consensus.cc:686] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15 [term 1 LEADER]: Becoming Leader. State: Replica: dea458ba48c149dfb26462a945579e15, State: Running, Role: LEADER
I20260925 09:32:00.079584 32488 consensus_queue.cc:237] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea458ba48c149dfb26462a945579e15" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 41427 } }
I20260925 09:32:00.080374 32478 sys_catalog.cc:564] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:32:00.082772 32490 sys_catalog.cc:455] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dea458ba48c149dfb26462a945579e15. Latest consensus state: current_term: 1 leader_uuid: "dea458ba48c149dfb26462a945579e15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea458ba48c149dfb26462a945579e15" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 41427 } } }
I20260925 09:32:00.083230 32490 sys_catalog.cc:458] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15 [sys.catalog]: This master's current role is: LEADER
I20260925 09:32:00.082527 32489 sys_catalog.cc:455] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dea458ba48c149dfb26462a945579e15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea458ba48c149dfb26462a945579e15" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 41427 } } }
I20260925 09:32:00.084115 32489 sys_catalog.cc:458] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15 [sys.catalog]: This master's current role is: LEADER
I20260925 09:32:00.085649 32481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:00.068098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55313 (local address 127.10.114.126:41427)
0925 09:32:00.068311 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:00.068320 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:00.068771 (+ 451us) server_negotiation.cc:407] Connection header received
0925 09:32:00.068976 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:00.068981 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:00.069050 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:00.069130 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:00.071149 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:00.072039 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:00.073854 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:00.074480 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:00.075343 (+ 863us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:00.077237 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:00.077242 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:00.077245 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:00.077273 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:00.080778 (+ 3505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:00.081680 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:00.081688 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:00.081695 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:00.081842 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:00.082429 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:00.082437 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:00.082440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:00.083051 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:00.083420 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:00.084881 (+ 1461us) server_negotiation.cc:299] Negotiation successful
0925 09:32:00.085111 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:32:00.091662 32493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:32:00.097386 32493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:32:00.106431 32493 catalog_manager.cc:1347] Generated new cluster ID: 798a19f64e8f4461be6d1e21838c441a
I20260925 09:32:00.106549 32493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:32:00.114866 32493 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:32:00.116150 32493 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:32:00.122880 32493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dea458ba48c149dfb26462a945579e15: Generated new TSK 0
I20260925 09:32:00.123603 32493 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:32:00.132707 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 32411
I20260925 09:32:00.134569 32473 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:32:00.223039 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 32411
W20260925 09:32:00.240229 32396 connection.cc:535] client connection to 127.10.114.126:41427 recv error: Network error: failed to read from TLS socket (remote: 127.10.114.126:41427): Connection reset by peer (error 104)
2026-09-25T09:32:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1937 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:32:00 dist-test-slave-9pj8 krb5kdc[32510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:32:00 dist-test-slave-9pj8 krb5kdc[32510](info): set up 2 sockets
Sep 25 09:32:00 dist-test-slave-9pj8 krb5kdc[32510](info): commencing operation
krb5kdc: starting...
W20260925 09:32:01.316766 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.001s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:32:01 dist-test-slave-9pj8 krb5kdc[32510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:32:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:32:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:32:01.551091 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:32875
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:35159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:32875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:32:01.864118 32523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:32:01.864391 32523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:32:01.864482 32523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:32:01.871400 32523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:32:01.871505 32523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:32:01.871569 32523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:32:01.871629 32523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:32:01.871672 32523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:32:01.881239 32523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:35159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:32875
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:32875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:32:01.882468 32523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:32:01.883733 32523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:32:01.896346 32528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:01.896265 32529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:01.896317 32531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:32:01.897147 32523 server_base.cc:1034] running on GCE node
I20260925 09:32:01.897831 32523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:32:01.899168 32523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:32:01.900420 32523 hybrid_clock.cc:648] HybridClock initialized: now 1790328721900362 us; error 50 us; skew 500 ppm
I20260925 09:32:01.900830 32523 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:32:01 dist-test-slave-9pj8 krb5kdc[32510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328721, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:32:01.909406 32523 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:32:01.910984 32523 webserver.cc:460] Webserver started at http://127.10.114.126:38455/ using document root <none> and password file <none>
I20260925 09:32:01.911700 32523 fs_manager.cc:359] Metadata directory not provided
I20260925 09:32:01.911865 32523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:32:01.912171 32523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:32:01.915158 32523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "054a207d4fda47e9bc773fbb4f9e1a39"
format_stamp: "Formatted at 2026-09-25 09:32:01 on dist-test-slave-9pj8"
server_key: "40adebc7c1c753dff6f51159f7103a0e"
server_key_iv: "5047d6ff91182a2478780dd241b8c608"
server_key_version: "encryptionkey@0"
I20260925 09:32:01.915999 32523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "054a207d4fda47e9bc773fbb4f9e1a39"
format_stamp: "Formatted at 2026-09-25 09:32:01 on dist-test-slave-9pj8"
server_key: "40adebc7c1c753dff6f51159f7103a0e"
server_key_iv: "5047d6ff91182a2478780dd241b8c608"
server_key_version: "encryptionkey@0"
I20260925 09:32:01.921152 32523 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260925 09:32:01.924549 32538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:32:01.925902 32523 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:32:01.926074 32523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "054a207d4fda47e9bc773fbb4f9e1a39"
format_stamp: "Formatted at 2026-09-25 09:32:01 on dist-test-slave-9pj8"
server_key: "40adebc7c1c753dff6f51159f7103a0e"
server_key_iv: "5047d6ff91182a2478780dd241b8c608"
server_key_version: "encryptionkey@0"
I20260925 09:32:01.926265 32523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:32:01.950894 32523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:32:01.954869 32523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:32:01.955093 32523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:32:01.973304 32523 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:32875
I20260925 09:32:01.973328 32589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:32875 every 8 connection(s)
I20260925 09:32:01.974653 32523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:32:01.978308 32590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:32:01.981747 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 32523
I20260925 09:32:01.981925 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:32:01.982441 10697 external_mini_cluster.cc:1428] Setting key 6a87c1edebed79f5dcdf3b73dd3a1024
I20260925 09:32:01.990372 32590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39: Bootstrap starting.
I20260925 09:32:01.993942 32590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39: Neither blocks nor log segments found. Creating new log.
I20260925 09:32:01.995004 32590 log.cc:824] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39: Log is configured to *not* fsync() on all Append() calls
I20260925 09:32:01.995496 32593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:01.984196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44801 (local address 127.10.114.126:32875)
0925 09:32:01.984926 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:01.984936 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:01.984974 (+ 38us) server_negotiation.cc:407] Connection header received
0925 09:32:01.986525 (+ 1551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:01.986545 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:01.986921 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:01.987219 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:01.987834 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:01.989182 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:01.990996 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:01.991638 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:01.991766 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:01.993156 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:01.993164 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:01.993176 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:32:01.993691 (+ 515us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:01.993761 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:32:01.993783 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:01.993906 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:01.994293 (+ 387us) server_negotiation.cc:299] Negotiation successful
0925 09:32:01.994486 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":359,"threads_started":1}
I20260925 09:32:01.998208 32590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39: No bootstrap required, opened a new log
I20260925 09:32:01.999485 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:32:02.010327 32590 raft_consensus.cc:348] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054a207d4fda47e9bc773fbb4f9e1a39" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 32875 } }
I20260925 09:32:02.011042 32590 raft_consensus.cc:374] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:32:02.011199 32590 raft_consensus.cc:729] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 054a207d4fda47e9bc773fbb4f9e1a39, State: Initialized, Role: FOLLOWER
I20260925 09:32:02.012188 32590 consensus_queue.cc:260] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054a207d4fda47e9bc773fbb4f9e1a39" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 32875 } }
I20260925 09:32:02.012569 32590 raft_consensus.cc:388] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:32:02.012737 32590 raft_consensus.cc:482] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:32:02.012933 32590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:32:02.014430 32593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:02.005444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42321 (local address 127.10.114.126:32875)
0925 09:32:02.005643 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:02.005651 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:02.006584 (+ 933us) server_negotiation.cc:407] Connection header received
0925 09:32:02.006667 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:02.006675 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:02.006746 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:02.006863 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:02.007539 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:02.008937 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:02.011071 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:02.011760 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:02.011886 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:02.013718 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:02.013723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:02.013726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:32:02.013743 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:02.013778 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:32:02.013796 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:02.013926 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:02.014095 (+ 169us) server_negotiation.cc:299] Negotiation successful
0925 09:32:02.014145 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:32:02.017261 32590 raft_consensus.cc:504] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054a207d4fda47e9bc773fbb4f9e1a39" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 32875 } }
I20260925 09:32:02.017992 32590 leader_election.cc:302] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 054a207d4fda47e9bc773fbb4f9e1a39; no voters:
I20260925 09:32:02.018662 32590 leader_election.cc:288] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:32:02.018780 32600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:32:02.020996 32590 sys_catalog.cc:564] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:32:02.025933 32600 raft_consensus.cc:686] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39 [term 1 LEADER]: Becoming Leader. State: Replica: 054a207d4fda47e9bc773fbb4f9e1a39, State: Running, Role: LEADER
I20260925 09:32:02.027029 32600 consensus_queue.cc:237] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054a207d4fda47e9bc773fbb4f9e1a39" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 32875 } }
I20260925 09:32:02.030062 32601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "054a207d4fda47e9bc773fbb4f9e1a39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054a207d4fda47e9bc773fbb4f9e1a39" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 32875 } } }
I20260925 09:32:02.030407 32601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39 [sys.catalog]: This master's current role is: LEADER
I20260925 09:32:02.030948 32602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 054a207d4fda47e9bc773fbb4f9e1a39. Latest consensus state: current_term: 1 leader_uuid: "054a207d4fda47e9bc773fbb4f9e1a39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054a207d4fda47e9bc773fbb4f9e1a39" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 32875 } } }
I20260925 09:32:02.031262 32602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39 [sys.catalog]: This master's current role is: LEADER
W20260925 09:32:02.036841 32615 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:32:02.036995 32615 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:32:02.037142 32614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:32:02.038434 32614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:32:02.046111 32614 catalog_manager.cc:1347] Generated new cluster ID: b98d0461f90c4220bf342a6efd0a3c6e
I20260925 09:32:02.046206 32614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:32:02.058290 32614 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:32:02.059605 32614 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:32:02.070266 32614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 054a207d4fda47e9bc773fbb4f9e1a39: Generated new TSK 0
I20260925 09:32:02.071205 32614 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:32:02.086589 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 32523
W20260925 09:32:02.088002 32585 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44801
W20260925 09:32:02.088687 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 32523: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:32:02.088831 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 32523
W20260925 09:32:02.105974 32508 connection.cc:535] client connection to 127.10.114.126:32875 recv error: Network error: failed to read from TLS socket (remote: 127.10.114.126:32875): Connection reset by peer (error 104)
2026-09-25T09:32:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1866 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:32:02 dist-test-slave-9pj8 krb5kdc[32621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:32:02 dist-test-slave-9pj8 krb5kdc[32621](info): set up 2 sockets
Sep 25 09:32:02 dist-test-slave-9pj8 krb5kdc[32621](info): commencing operation
krb5kdc: starting...
W20260925 09:32:03.164319 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:32:03 dist-test-slave-9pj8 krb5kdc[32621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:32:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:32:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:32:03.398999 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:33953
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:39027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:33953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:32:03.706720 32634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:32:03.707062 32634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:32:03.707217 32634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:32:03.714342 32634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:32:03.714478 32634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:32:03.714586 32634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:32:03.714684 32634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:32:03.714754 32634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:32:03.725142 32634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:39027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:33953
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:33953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:32:03.726745 32634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:32:03.728536 32634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:32:03.740370 32639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:03.740322 32640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:03.740391 32642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:32:03.741149 32634 server_base.cc:1034] running on GCE node
I20260925 09:32:03.741811 32634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:32:03.743021 32634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:32:03.744215 32634 hybrid_clock.cc:648] HybridClock initialized: now 1790328723744182 us; error 22 us; skew 500 ppm
I20260925 09:32:03.744567 32634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:32:03 dist-test-slave-9pj8 krb5kdc[32621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328723, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:32:03.747833 32634 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:32:03.749248 32634 webserver.cc:460] Webserver started at http://127.10.114.126:45691/ using document root <none> and password file <none>
I20260925 09:32:03.749892 32634 fs_manager.cc:359] Metadata directory not provided
I20260925 09:32:03.750010 32634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:32:03.750300 32634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:32:03.753155 32634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "9bb2fd10f1d24e01a771c143f36d82a3"
format_stamp: "Formatted at 2026-09-25 09:32:03 on dist-test-slave-9pj8"
server_key: "367bde6efe044f390e232d49c066d352"
server_key_iv: "1e6308733c5bda6efb1e7f6973b7e23a"
server_key_version: "encryptionkey@0"
I20260925 09:32:03.753888 32634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "9bb2fd10f1d24e01a771c143f36d82a3"
format_stamp: "Formatted at 2026-09-25 09:32:03 on dist-test-slave-9pj8"
server_key: "367bde6efe044f390e232d49c066d352"
server_key_iv: "1e6308733c5bda6efb1e7f6973b7e23a"
server_key_version: "encryptionkey@0"
I20260925 09:32:03.758668 32634 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260925 09:32:03.761837 32649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:32:03.763008 32634 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260925 09:32:03.763173 32634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "9bb2fd10f1d24e01a771c143f36d82a3"
format_stamp: "Formatted at 2026-09-25 09:32:03 on dist-test-slave-9pj8"
server_key: "367bde6efe044f390e232d49c066d352"
server_key_iv: "1e6308733c5bda6efb1e7f6973b7e23a"
server_key_version: "encryptionkey@0"
I20260925 09:32:03.763340 32634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:32:03.769280 32634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:32:03.773011 32634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:32:03.773263 32634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:32:03.791628 32634 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:33953
I20260925 09:32:03.791666 32700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:33953 every 8 connection(s)
I20260925 09:32:03.792999 32634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:32:03.797137 32701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:32:03.798622 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 32634
I20260925 09:32:03.798804 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:32:03.799347 10697 external_mini_cluster.cc:1428] Setting key 1c51f444d42e651324090763ea4cf978
I20260925 09:32:03.809254 32704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:03.801494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43143 (local address 127.10.114.126:33953)
0925 09:32:03.802507 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:03.802523 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:03.802923 (+ 400us) server_negotiation.cc:407] Connection header received
0925 09:32:03.805334 (+ 2411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:03.805362 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:03.805801 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:03.806267 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:03.807268 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:03.807276 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:03.807294 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:32:03.807310 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:03.807379 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:32:03.807410 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:03.807641 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:03.807795 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 09:32:03.808003 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"thread_start_us":491,"threads_started":1}
I20260925 09:32:03.810956 32701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3: Bootstrap starting.
I20260925 09:32:03.812743 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:32:03.814162 32701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3: Neither blocks nor log segments found. Creating new log.
I20260925 09:32:03.815095 32701 log.cc:824] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3: Log is configured to *not* fsync() on all Append() calls
I20260925 09:32:03.818220 32701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3: No bootstrap required, opened a new log
I20260925 09:32:03.819419 32704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:03.817063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40047 (local address 127.10.114.126:33953)
0925 09:32:03.817279 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:03.817284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:03.817646 (+ 362us) server_negotiation.cc:407] Connection header received
0925 09:32:03.817768 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:03.817775 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:03.817828 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:03.817939 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:03.818784 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:03.818789 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:03.818795 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:32:03.818815 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:03.818892 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:32:03.818916 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:03.819041 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:03.819109 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 09:32:03.819161 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260925 09:32:03.825814 32701 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb2fd10f1d24e01a771c143f36d82a3" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33953 } }
I20260925 09:32:03.826251 32701 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:32:03.826366 32701 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bb2fd10f1d24e01a771c143f36d82a3, State: Initialized, Role: FOLLOWER
I20260925 09:32:03.827011 32701 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb2fd10f1d24e01a771c143f36d82a3" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33953 } }
I20260925 09:32:03.827220 32701 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:32:03.827316 32701 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:32:03.827442 32701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:32:03.829548 32701 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb2fd10f1d24e01a771c143f36d82a3" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33953 } }
I20260925 09:32:03.829999 32701 leader_election.cc:302] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bb2fd10f1d24e01a771c143f36d82a3; no voters:
I20260925 09:32:03.830510 32701 leader_election.cc:288] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:32:03.830649 32711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:32:03.831256 32711 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3 [term 1 LEADER]: Becoming Leader. State: Replica: 9bb2fd10f1d24e01a771c143f36d82a3, State: Running, Role: LEADER
I20260925 09:32:03.831735 32711 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb2fd10f1d24e01a771c143f36d82a3" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33953 } }
I20260925 09:32:03.832566 32701 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:32:03.834327 32712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bb2fd10f1d24e01a771c143f36d82a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb2fd10f1d24e01a771c143f36d82a3" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33953 } } }
I20260925 09:32:03.834761 32712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3 [sys.catalog]: This master's current role is: LEADER
I20260925 09:32:03.834671 32713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bb2fd10f1d24e01a771c143f36d82a3. Latest consensus state: current_term: 1 leader_uuid: "9bb2fd10f1d24e01a771c143f36d82a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb2fd10f1d24e01a771c143f36d82a3" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33953 } } }
I20260925 09:32:03.835132 32713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3 [sys.catalog]: This master's current role is: LEADER
I20260925 09:32:03.837663 32716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:32:03.844116 32716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:32:03.853261 32716 catalog_manager.cc:1347] Generated new cluster ID: f4b730981b3e42a288293fb14f4babd9
I20260925 09:32:03.853372 32716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:32:03.868884 32716 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:32:03.869364 32716 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:32:03.875375 32716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9bb2fd10f1d24e01a771c143f36d82a3: Generated new TSK 0
I20260925 09:32:03.876315 32716 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:32:03.886446 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 32634
W20260925 09:32:03.887493 32696 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43143
W20260925 09:32:03.888175 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 32634: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:32:03.888319 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 32634
2026-09-25T09:32:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1800 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:32:03 dist-test-slave-9pj8 krb5kdc[32732](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:32:03 dist-test-slave-9pj8 krb5kdc[32732](info): set up 2 sockets
Sep 25 09:32:03 dist-test-slave-9pj8 krb5kdc[32732](info): commencing operation
krb5kdc: starting...
W20260925 09:32:04.980674 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:32:05 dist-test-slave-9pj8 krb5kdc[32732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:32:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:32:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:32:05.237085 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:39191
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:36739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:39191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:32:05.555810 32745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:32:05.556094 32745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:32:05.556248 32745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:32:05.563158 32745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:32:05.563271 32745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:32:05.563369 32745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:32:05.563462 32745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:32:05.563549 32745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:32:05.573004 32745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:36739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:39191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:39191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:32:05.574317 32745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:32:05.575640 32745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:32:05.587388 32750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:05.587529 32753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:05.587564 32751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:32:05.588694 32745 server_base.cc:1034] running on GCE node
I20260925 09:32:05.589475 32745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:32:05.590953 32745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:32:05.592180 32745 hybrid_clock.cc:648] HybridClock initialized: now 1790328725592152 us; error 58 us; skew 500 ppm
I20260925 09:32:05.592557 32745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:32:05 dist-test-slave-9pj8 krb5kdc[32732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328725, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:32:05.595557 32745 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:32:05.597114 32745 webserver.cc:460] Webserver started at http://127.10.114.126:37517/ using document root <none> and password file <none>
I20260925 09:32:05.597805 32745 fs_manager.cc:359] Metadata directory not provided
I20260925 09:32:05.597959 32745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:32:05.598263 32745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:32:05.601244 32745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "e1133e7e71f94b6db762aff4ed4bfc03"
format_stamp: "Formatted at 2026-09-25 09:32:05 on dist-test-slave-9pj8"
server_key: "c7afa5921a953d87832c84773284d87d"
server_key_iv: "d127ddc24cbad432b91578ed9d1e387c"
server_key_version: "encryptionkey@0"
I20260925 09:32:05.602012 32745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "e1133e7e71f94b6db762aff4ed4bfc03"
format_stamp: "Formatted at 2026-09-25 09:32:05 on dist-test-slave-9pj8"
server_key: "c7afa5921a953d87832c84773284d87d"
server_key_iv: "d127ddc24cbad432b91578ed9d1e387c"
server_key_version: "encryptionkey@0"
I20260925 09:32:05.607092 32745 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260925 09:32:05.610895 32760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:32:05.612373 32745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:32:05.612565 32745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "e1133e7e71f94b6db762aff4ed4bfc03"
format_stamp: "Formatted at 2026-09-25 09:32:05 on dist-test-slave-9pj8"
server_key: "c7afa5921a953d87832c84773284d87d"
server_key_iv: "d127ddc24cbad432b91578ed9d1e387c"
server_key_version: "encryptionkey@0"
I20260925 09:32:05.612761 32745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:32:05.629388 32745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:32:05.633212 32745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:32:05.633510 32745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:32:05.651865 32745 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:39191
I20260925 09:32:05.651947 343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:39191 every 8 connection(s)
I20260925 09:32:05.653157 32745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:32:05.656766 344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:32:05.660427 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 32745
I20260925 09:32:05.660616 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:32:05.661058 10697 external_mini_cluster.cc:1428] Setting key ed858fb830bf17ada906ae5d18aef257
I20260925 09:32:05.669456 344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03: Bootstrap starting.
I20260925 09:32:05.673072 344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03: Neither blocks nor log segments found. Creating new log.
I20260925 09:32:05.674350 344 log.cc:824] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03: Log is configured to *not* fsync() on all Append() calls
Sep 25 09:32:05 dist-test-slave-9pj8 krb5kdc[32732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:32:05.677348 344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03: No bootstrap required, opened a new log
I20260925 09:32:05.684978 347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:05.662878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44859 (local address 127.10.114.126:39191)
0925 09:32:05.663666 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:05.663682 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:05.663736 (+ 54us) server_negotiation.cc:407] Connection header received
0925 09:32:05.665405 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:05.665425 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:05.665816 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:05.666133 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:05.667960 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:05.669230 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:05.670990 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:05.671709 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:05.671951 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:05.676715 (+ 4764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:05.676739 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:05.676758 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:05.676799 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:05.680800 (+ 4001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:05.681377 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:05.681385 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:05.681393 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:05.681598 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:05.681944 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:05.681950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:05.681954 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:05.682617 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:05.682924 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:05.683219 (+ 295us) server_negotiation.cc:299] Negotiation successful
0925 09:32:05.683653 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":356,"threads_started":1}
I20260925 09:32:05.687479 344 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1133e7e71f94b6db762aff4ed4bfc03" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39191 } }
I20260925 09:32:05.687897 344 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:32:05.688040 344 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1133e7e71f94b6db762aff4ed4bfc03, State: Initialized, Role: FOLLOWER
I20260925 09:32:05.688659 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:32:05.688608 344 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1133e7e71f94b6db762aff4ed4bfc03" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39191 } }
I20260925 09:32:05.688880 344 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:32:05.689035 344 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:32:05.689208 344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:32:05.691238 344 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1133e7e71f94b6db762aff4ed4bfc03" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39191 } }
I20260925 09:32:05.691690 344 leader_election.cc:302] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1133e7e71f94b6db762aff4ed4bfc03; no voters:
I20260925 09:32:05.692288 344 leader_election.cc:288] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:32:05.692430 353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:32:05.694087 353 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03 [term 1 LEADER]: Becoming Leader. State: Replica: e1133e7e71f94b6db762aff4ed4bfc03, State: Running, Role: LEADER
I20260925 09:32:05.694329 344 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:32:05.694643 353 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1133e7e71f94b6db762aff4ed4bfc03" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39191 } }
I20260925 09:32:05.701287 354 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1133e7e71f94b6db762aff4ed4bfc03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1133e7e71f94b6db762aff4ed4bfc03" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39191 } } }
I20260925 09:32:05.701632 354 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03 [sys.catalog]: This master's current role is: LEADER
I20260925 09:32:05.702097 355 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1133e7e71f94b6db762aff4ed4bfc03. Latest consensus state: current_term: 1 leader_uuid: "e1133e7e71f94b6db762aff4ed4bfc03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1133e7e71f94b6db762aff4ed4bfc03" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39191 } } }
I20260925 09:32:05.703449 355 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03 [sys.catalog]: This master's current role is: LEADER
I20260925 09:32:05.707849 361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:32:05.715546 347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:05.693413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47787 (local address 127.10.114.126:39191)
0925 09:32:05.693605 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:05.693612 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:05.693723 (+ 111us) server_negotiation.cc:407] Connection header received
0925 09:32:05.693967 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:05.693971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:05.694062 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:05.694141 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:05.696535 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:05.697986 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:05.700399 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:05.701351 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:05.701492 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:05.705367 (+ 3875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:05.705376 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:05.705383 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:05.705443 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:05.711508 (+ 6065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:05.712499 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:05.712507 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:05.712512 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:05.712682 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:05.713142 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:05.713146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:05.713149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:05.713590 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:05.713843 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:05.714993 (+ 1150us) server_negotiation.cc:299] Negotiation successful
0925 09:32:05.715199 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:32:05.716401 361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:32:05.725198 361 catalog_manager.cc:1347] Generated new cluster ID: 20128c73af624336a5fe0507cf360c09
I20260925 09:32:05.725301 361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:32:05.744153 361 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:32:05.745560 361 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:32:05.751101 361 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1133e7e71f94b6db762aff4ed4bfc03: Generated new TSK 0
I20260925 09:32:05.751904 361 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:32:05.759765 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 32745
I20260925 09:32:05.760912 339 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:32:05.849238 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 32745
2026-09-25T09:32:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1959 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:32:05 dist-test-slave-9pj8 krb5kdc[377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:32:05 dist-test-slave-9pj8 krb5kdc[377](info): set up 2 sockets
Sep 25 09:32:05 dist-test-slave-9pj8 krb5kdc[377](info): commencing operation
krb5kdc: starting...
W20260925 09:32:06.940469 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:32:07 dist-test-slave-9pj8 krb5kdc[377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:32:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:32:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:32:07.183193 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:44037
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:39181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:44037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:32:07.505460 390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:32:07.505812 390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:32:07.506011 390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:32:07.513218 390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:32:07.513375 390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:32:07.513494 390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:32:07.513613 390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:32:07.513697 390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:32:07.523289 390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:39181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:44037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:44037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:32:07.524776 390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:32:07.526113 390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:32:07.537662 396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:07.537698 395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:07.537611 398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:32:07.538769 390 server_base.cc:1034] running on GCE node
I20260925 09:32:07.539561 390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:32:07.540969 390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:32:07.542163 390 hybrid_clock.cc:648] HybridClock initialized: now 1790328727542112 us; error 45 us; skew 500 ppm
I20260925 09:32:07.542534 390 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:32:07 dist-test-slave-9pj8 krb5kdc[377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328727, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:32:07.550042 390 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:32:07.551492 390 webserver.cc:460] Webserver started at http://127.10.114.126:35933/ using document root <none> and password file <none>
I20260925 09:32:07.552239 390 fs_manager.cc:359] Metadata directory not provided
I20260925 09:32:07.552340 390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:32:07.552606 390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:32:07.555379 390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "37ebd624a4b1458889a24500caecbc94"
format_stamp: "Formatted at 2026-09-25 09:32:07 on dist-test-slave-9pj8"
server_key: "488c6af8a5c240781370d64b19f95b51"
server_key_iv: "19a1c9d79d86a5ceaa03c5a9e1636514"
server_key_version: "encryptionkey@0"
I20260925 09:32:07.556113 390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "37ebd624a4b1458889a24500caecbc94"
format_stamp: "Formatted at 2026-09-25 09:32:07 on dist-test-slave-9pj8"
server_key: "488c6af8a5c240781370d64b19f95b51"
server_key_iv: "19a1c9d79d86a5ceaa03c5a9e1636514"
server_key_version: "encryptionkey@0"
I20260925 09:32:07.561021 390 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260925 09:32:07.564309 405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:32:07.565579 390 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 09:32:07.565757 390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "37ebd624a4b1458889a24500caecbc94"
format_stamp: "Formatted at 2026-09-25 09:32:07 on dist-test-slave-9pj8"
server_key: "488c6af8a5c240781370d64b19f95b51"
server_key_iv: "19a1c9d79d86a5ceaa03c5a9e1636514"
server_key_version: "encryptionkey@0"
I20260925 09:32:07.565948 390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:32:07.587956 390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:32:07.592331 390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:32:07.592633 390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:32:07.610196 390 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:44037
I20260925 09:32:07.610219 456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:44037 every 8 connection(s)
I20260925 09:32:07.611558 390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:32:07.615272 457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:32:07.616156 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 390
I20260925 09:32:07.616341 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:32:07.616799 10697 external_mini_cluster.cc:1428] Setting key 62a640d28fe86a52395afc6133d3717b
I20260925 09:32:07.629220 457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94: Bootstrap starting.
I20260925 09:32:07.631807 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:07.618686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57279 (local address 127.10.114.126:44037)
0925 09:32:07.619548 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:07.619564 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:07.619623 (+ 59us) server_negotiation.cc:407] Connection header received
0925 09:32:07.622029 (+ 2406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:07.622056 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:07.622561 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:07.622975 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:07.623605 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:07.625322 (+ 1717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:07.626677 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:07.627687 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:07.627894 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:07.629101 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:07.629112 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:07.629127 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:32:07.629802 (+ 675us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:07.629887 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:32:07.629917 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:07.630027 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:07.630253 (+ 226us) server_negotiation.cc:299] Negotiation successful
0925 09:32:07.630536 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":413,"threads_started":1}
I20260925 09:32:07.632800 457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94: Neither blocks nor log segments found. Creating new log.
I20260925 09:32:07.634090 457 log.cc:824] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94: Log is configured to *not* fsync() on all Append() calls
I20260925 09:32:07.635550 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:32:07.636883 457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94: No bootstrap required, opened a new log
I20260925 09:32:07.644665 457 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ebd624a4b1458889a24500caecbc94" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44037 } }
I20260925 09:32:07.644990 457 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:32:07.645129 457 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37ebd624a4b1458889a24500caecbc94, State: Initialized, Role: FOLLOWER
I20260925 09:32:07.645689 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:07.639630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37073 (local address 127.10.114.126:44037)
0925 09:32:07.639825 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:07.639832 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:07.640186 (+ 354us) server_negotiation.cc:407] Connection header received
0925 09:32:07.640406 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:07.640410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:07.640465 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:07.640576 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:07.641065 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:07.641764 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:07.643023 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:07.643477 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:07.643640 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:07.644814 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:07.644819 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:07.644822 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:32:07.644838 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:07.644880 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:32:07.644896 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:07.645018 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:07.645190 (+ 172us) server_negotiation.cc:299] Negotiation successful
0925 09:32:07.645288 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 09:32:07.645809 457 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ebd624a4b1458889a24500caecbc94" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44037 } }
I20260925 09:32:07.646148 457 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:32:07.646317 457 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:32:07.646507 457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:32:07.648880 457 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ebd624a4b1458889a24500caecbc94" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44037 } }
I20260925 09:32:07.649294 457 leader_election.cc:302] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37ebd624a4b1458889a24500caecbc94; no voters:
I20260925 09:32:07.649901 457 leader_election.cc:288] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:32:07.650082 467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:32:07.651012 467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94 [term 1 LEADER]: Becoming Leader. State: Replica: 37ebd624a4b1458889a24500caecbc94, State: Running, Role: LEADER
I20260925 09:32:07.651541 467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ebd624a4b1458889a24500caecbc94" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44037 } }
I20260925 09:32:07.652091 457 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:32:07.654340 468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37ebd624a4b1458889a24500caecbc94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ebd624a4b1458889a24500caecbc94" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44037 } } }
I20260925 09:32:07.654760 468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94 [sys.catalog]: This master's current role is: LEADER
I20260925 09:32:07.654691 469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37ebd624a4b1458889a24500caecbc94. Latest consensus state: current_term: 1 leader_uuid: "37ebd624a4b1458889a24500caecbc94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ebd624a4b1458889a24500caecbc94" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 44037 } } }
I20260925 09:32:07.655220 469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94 [sys.catalog]: This master's current role is: LEADER
I20260925 09:32:07.656101 474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:32:07.661681 474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:32:07.670892 474 catalog_manager.cc:1347] Generated new cluster ID: 696163d3eaac4527b3dac46fa23c526f
I20260925 09:32:07.671018 474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:32:07.685575 474 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:32:07.686803 474 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:32:07.696745 474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37ebd624a4b1458889a24500caecbc94: Generated new TSK 0
I20260925 09:32:07.697563 474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:32:07.709707 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 390
W20260925 09:32:07.711027 452 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57279
W20260925 09:32:07.711951 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 390: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:32:07.712126 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 390
2026-09-25T09:32:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1862 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:32:07 dist-test-slave-9pj8 krb5kdc[488](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:32:07 dist-test-slave-9pj8 krb5kdc[488](info): set up 2 sockets
Sep 25 09:32:07 dist-test-slave-9pj8 krb5kdc[488](info): commencing operation
krb5kdc: starting...
W20260925 09:32:08.836668 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:32:09 dist-test-slave-9pj8 krb5kdc[488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:32:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:32:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:32:09.099694 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:33203
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:39041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:33203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:32:09.425603 501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:32:09.425901 501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:32:09.425990 501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:32:09.433229 501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:32:09.433342 501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:32:09.433386 501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:32:09.433445 501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:32:09.433488 501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:32:09.443305 501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:39041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:33203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:33203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:32:09.444639 501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:32:09.445916 501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:32:09.457544 509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:09.457643 506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:09.457620 507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:32:09.458671 501 server_base.cc:1034] running on GCE node
I20260925 09:32:09.459556 501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:32:09.461253 501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:32:09.462592 501 hybrid_clock.cc:648] HybridClock initialized: now 1790328729462541 us; error 31 us; skew 500 ppm
I20260925 09:32:09.463001 501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:32:09 dist-test-slave-9pj8 krb5kdc[488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328729, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:32:09.466277 501 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:32:09.467851 501 webserver.cc:460] Webserver started at http://127.10.114.126:37363/ using document root <none> and password file <none>
I20260925 09:32:09.468570 501 fs_manager.cc:359] Metadata directory not provided
I20260925 09:32:09.468734 501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:32:09.469055 501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:32:09.472214 501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "fa5571b3b78541778d30083882acf364"
format_stamp: "Formatted at 2026-09-25 09:32:09 on dist-test-slave-9pj8"
server_key: "6a446a9cc93eecdca8145fd0bd601f50"
server_key_iv: "5269c0c6037e651bf8ab1ad5cca8660c"
server_key_version: "encryptionkey@0"
I20260925 09:32:09.473044 501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "fa5571b3b78541778d30083882acf364"
format_stamp: "Formatted at 2026-09-25 09:32:09 on dist-test-slave-9pj8"
server_key: "6a446a9cc93eecdca8145fd0bd601f50"
server_key_iv: "5269c0c6037e651bf8ab1ad5cca8660c"
server_key_version: "encryptionkey@0"
I20260925 09:32:09.478515 501 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260925 09:32:09.482247 516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:32:09.483618 501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 09:32:09.483870 501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "fa5571b3b78541778d30083882acf364"
format_stamp: "Formatted at 2026-09-25 09:32:09 on dist-test-slave-9pj8"
server_key: "6a446a9cc93eecdca8145fd0bd601f50"
server_key_iv: "5269c0c6037e651bf8ab1ad5cca8660c"
server_key_version: "encryptionkey@0"
I20260925 09:32:09.484091 501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:32:09.489754 501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:32:09.494469 501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:32:09.494791 501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:32:09.514744 501 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:33203
I20260925 09:32:09.514766 567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:33203 every 8 connection(s)
I20260925 09:32:09.516295 501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:32:09.520092 568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:32:09.524544 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 501
I20260925 09:32:09.524744 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:32:09.525239 10697 external_mini_cluster.cc:1428] Setting key 406e40b6e314c6f6823e75fa974a357a
I20260925 09:32:09.531086 568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364: Bootstrap starting.
I20260925 09:32:09.533527 571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:09.527124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46045 (local address 127.10.114.126:33203)
0925 09:32:09.528068 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:09.528079 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:09.528128 (+ 49us) server_negotiation.cc:407] Connection header received
0925 09:32:09.530091 (+ 1963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:09.530114 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:09.530459 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:09.530802 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:09.531924 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:09.531931 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:09.531943 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:32:09.531959 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:09.532028 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:32:09.532049 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:09.532308 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:09.532459 (+ 151us) server_negotiation.cc:299] Negotiation successful
0925 09:32:09.532642 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":405,"threads_started":1}
I20260925 09:32:09.534356 568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364: Neither blocks nor log segments found. Creating new log.
I20260925 09:32:09.535434 568 log.cc:824] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364: Log is configured to *not* fsync() on all Append() calls
I20260925 09:32:09.537412 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:32:09.538394 568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364: No bootstrap required, opened a new log
I20260925 09:32:09.544948 571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:09.541920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34717 (local address 127.10.114.126:33203)
0925 09:32:09.542144 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:09.542154 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:09.542728 (+ 574us) server_negotiation.cc:407] Connection header received
0925 09:32:09.542939 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:09.542945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:09.543009 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:09.543143 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:09.544169 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:09.544176 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:09.544181 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:32:09.544210 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:09.544248 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:32:09.544264 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:09.544396 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:09.544571 (+ 175us) server_negotiation.cc:299] Negotiation successful
0925 09:32:09.544651 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 09:32:09.546645 568 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa5571b3b78541778d30083882acf364" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33203 } }
I20260925 09:32:09.547008 568 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:32:09.547115 568 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa5571b3b78541778d30083882acf364, State: Initialized, Role: FOLLOWER
I20260925 09:32:09.547701 568 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa5571b3b78541778d30083882acf364" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33203 } }
I20260925 09:32:09.548012 568 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:32:09.548162 568 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:32:09.548341 568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:32:09.550426 568 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa5571b3b78541778d30083882acf364" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33203 } }
I20260925 09:32:09.550905 568 leader_election.cc:302] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa5571b3b78541778d30083882acf364; no voters:
I20260925 09:32:09.551467 568 leader_election.cc:288] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:32:09.551617 578 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:32:09.552872 578 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364 [term 1 LEADER]: Becoming Leader. State: Replica: fa5571b3b78541778d30083882acf364, State: Running, Role: LEADER
I20260925 09:32:09.553350 578 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa5571b3b78541778d30083882acf364" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33203 } }
I20260925 09:32:09.553570 568 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:32:09.555958 579 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa5571b3b78541778d30083882acf364" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa5571b3b78541778d30083882acf364" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33203 } } }
I20260925 09:32:09.556528 579 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364 [sys.catalog]: This master's current role is: LEADER
I20260925 09:32:09.556226 580 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa5571b3b78541778d30083882acf364. Latest consensus state: current_term: 1 leader_uuid: "fa5571b3b78541778d30083882acf364" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa5571b3b78541778d30083882acf364" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 33203 } } }
I20260925 09:32:09.556654 580 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364 [sys.catalog]: This master's current role is: LEADER
I20260925 09:32:09.557669 585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:32:09.563544 585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:32:09.571544 585 catalog_manager.cc:1347] Generated new cluster ID: 228beb572f6746aa9e9e3e4baa30210d
I20260925 09:32:09.571662 585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:32:09.606014 585 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:32:09.606470 585 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:32:09.616067 585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa5571b3b78541778d30083882acf364: Generated new TSK 0
I20260925 09:32:09.616875 585 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:32:09.628827 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 501
W20260925 09:32:09.629855 563 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46045
W20260925 09:32:09.630584 10697 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 501: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:32:09.630710 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 501
2026-09-25T09:32:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1918 ms)
[----------] 18 tests from AuthTokenIssuingTest (33910 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:32:09 dist-test-slave-9pj8 krb5kdc[599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:32:09 dist-test-slave-9pj8 krb5kdc[599](info): set up 2 sockets
Sep 25 09:32:09 dist-test-slave-9pj8 krb5kdc[599](info): commencing operation
krb5kdc: starting...
W20260925 09:32:10.732518 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:32:10 dist-test-slave-9pj8 krb5kdc[599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:32:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:32:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:32:10.989480 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:39341
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:39341,127.10.114.125:33645,127.10.114.124:35601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:32:11.314184 612 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 09:32:11.314471 612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:32:11.314549 612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:32:11.314627 612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:32:11.321578 612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:32:11.321652 612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:32:11.321705 612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:32:11.321746 612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:32:11.321805 612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:32:11.330835 612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:39341,127.10.114.125:33645,127.10.114.124:35601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:39341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:32:11.332026 612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:32:11.333266 612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:32:11.345399 617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:11.345364 618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:11.345443 620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:32:11.346290 612 server_base.cc:1034] running on GCE node
I20260925 09:32:11.347103 612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:32:11.348560 612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:32:11.349781 612 hybrid_clock.cc:648] HybridClock initialized: now 1790328731349708 us; error 54 us; skew 500 ppm
I20260925 09:32:11.350106 612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:32:11 dist-test-slave-9pj8 krb5kdc[599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328731, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:32:11.353169 612 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:32:11.354604 612 webserver.cc:460] Webserver started at http://127.10.114.126:44527/ using document root <none> and password file <none>
I20260925 09:32:11.355319 612 fs_manager.cc:359] Metadata directory not provided
I20260925 09:32:11.355463 612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:32:11.355825 612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:32:11.358646 612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "d5b781c4737846ffa408dad0086a9f68"
format_stamp: "Formatted at 2026-09-25 09:32:11 on dist-test-slave-9pj8"
server_key: "12f0b3c3c7ffa0a32651005feec50b5b"
server_key_iv: "f60e56268fec1bc72977f48aabb8a025"
server_key_version: "encryptionkey@0"
I20260925 09:32:11.359462 612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "d5b781c4737846ffa408dad0086a9f68"
format_stamp: "Formatted at 2026-09-25 09:32:11 on dist-test-slave-9pj8"
server_key: "12f0b3c3c7ffa0a32651005feec50b5b"
server_key_iv: "f60e56268fec1bc72977f48aabb8a025"
server_key_version: "encryptionkey@0"
I20260925 09:32:11.364300 612 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 09:32:11.367601 627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:32:11.368921 612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:32:11.369102 612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "d5b781c4737846ffa408dad0086a9f68"
format_stamp: "Formatted at 2026-09-25 09:32:11 on dist-test-slave-9pj8"
server_key: "12f0b3c3c7ffa0a32651005feec50b5b"
server_key_iv: "f60e56268fec1bc72977f48aabb8a025"
server_key_version: "encryptionkey@0"
I20260925 09:32:11.369315 612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:32:11.389490 612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:32:11.393500 612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:32:11.393762 612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:32:11.412833 612 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:39341
I20260925 09:32:11.412868 678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:39341 every 8 connection(s)
I20260925 09:32:11.414151 612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:32:11.417712 679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:32:11.423229 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 612
I20260925 09:32:11.423435 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:32:11.424279 10697 external_mini_cluster.cc:1428] Setting key 38da99e9edd58a890c7b2a75c4ef2171
I20260925 09:32:11.425009 679 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:32:11 dist-test-slave-9pj8 krb5kdc[599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328731, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:32:11.450958 682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:11.429283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54412 (local address 127.10.114.126:39341)
0925 09:32:11.431429 (+ 2146us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:11.431436 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:11.431481 (+ 45us) server_negotiation.cc:407] Connection header received
0925 09:32:11.433723 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:11.433729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:11.433950 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:11.434122 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:11.435442 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:11.437456 (+ 2014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:11.438851 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:11.439843 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:11.440028 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:11.443648 (+ 3620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:11.443656 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:11.443669 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:11.443701 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:11.446534 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:11.447513 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:11.447519 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:11.447522 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:11.447662 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:11.448233 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:11.448236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:11.448238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:11.448702 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:11.448854 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:11.449626 (+ 772us) server_negotiation.cc:299] Negotiation successful
0925 09:32:11.449880 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1930,"spinlock_wait_cycles":68352,"thread_start_us":1677,"threads_started":1}
I20260925 09:32:11.451015 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:11.429283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39341 (local address 127.0.0.1:54412)
0925 09:32:11.430265 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:11.430315 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 09:32:11.433310 (+ 2995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:32:11.434211 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:32:11.434221 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:32:11.434846 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:32:11.435206 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:11.435224 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:11.437616 (+ 2392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:11.437620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:11.438687 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:11.438698 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:11.440048 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:11.440053 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:11.440176 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:11.440859 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:32:11.440885 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:32:11.443422 (+ 2537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:32:11.446718 (+ 3296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:11.446735 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:11.446750 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:11.447330 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:11.447851 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:11.447858 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:11.447864 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:11.448113 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:11.448878 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:32:11.448887 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:32:11.449177 (+ 290us) client_negotiation.cc:769] Sending connection context
0925 09:32:11.449490 (+ 313us) client_negotiation.cc:240] Negotiation successful
0925 09:32:11.450099 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":78,"thread_start_us":429,"threads_started":1}
WARNING: no policy specified for kudu/127.10.114.125@KRBTEST.COM; defaulting to no policy
I20260925 09:32:11.454027 679 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.125@KRBTEST.COM" created.
I20260925 09:32:11.455230 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:11.454750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33645 (local address 127.0.0.1:45252)
0925 09:32:11.454948 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:11.455076 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260925 09:32:11.455536 630 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33645: connect: Connection refused (error 111)
I20260925 09:32:11.457352 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:11.456862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33645 (local address 127.0.0.1:45262)
0925 09:32:11.457084 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:11.457172 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260925 09:32:11.457587 679 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.125:33645: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33645: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.125.keytab.
Entry for principal kudu/127.10.114.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.125.keytab.
I20260925 09:32:11.484532 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.125:33645
--webserver_interface=127.10.114.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.125.keytab
--principal=kudu/127.10.114.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:39341,127.10.114.125:33645,127.10.114.124:35601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:32:11.506961 679 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } attempt: 1
I20260925 09:32:11.508271 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:11.507721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33645 (local address 127.0.0.1:45278)
0925 09:32:11.507996 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:11.508144 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260925 09:32:11.509337 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:11.509006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33645 (local address 127.0.0.1:45292)
0925 09:32:11.509137 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:11.509218 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260925 09:32:11.509656 679 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.125:33645: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33645: connect: Connection refused (error 111)
I20260925 09:32:11.577945 679 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } attempt: 2
I20260925 09:32:11.579229 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:11.578698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33645 (local address 127.0.0.1:45300)
0925 09:32:11.578967 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:11.579066 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260925 09:32:11.580289 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:11.579940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33645 (local address 127.0.0.1:45312)
0925 09:32:11.580071 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:11.580157 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260925 09:32:11.580533 679 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.125:33645: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33645: connect: Connection refused (error 111)
I20260925 09:32:11.671885 679 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } attempt: 3
I20260925 09:32:11.673122 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:11.672704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33645 (local address 127.0.0.1:45322)
0925 09:32:11.672894 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:11.672990 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260925 09:32:11.674278 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:11.673877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33645 (local address 127.0.0.1:45324)
0925 09:32:11.674025 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:11.674097 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260925 09:32:11.674489 679 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.125:33645: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33645: connect: Connection refused (error 111)
W20260925 09:32:11.811659 686 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 09:32:11.812100 686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:32:11.812239 686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:32:11.812376 686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260925 09:32:11.817766 679 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } attempt: 4
I20260925 09:32:11.819278 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:11.818693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33645 (local address 127.0.0.1:45336)
0925 09:32:11.819028 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:11.819137 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189}
W20260925 09:32:11.820101 686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:32:11.820214 686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:32:11.820333 686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:32:11.820437 686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
I20260925 09:32:11.820472 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:11.820028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33645 (local address 127.0.0.1:45344)
0925 09:32:11.820183 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:11.820304 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260925 09:32:11.820547 686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 09:32:11.820724 679 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.125:33645: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33645: connect: Connection refused (error 111)
I20260925 09:32:11.830361 686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:39341,127.10.114.125:33645,127.10.114.124:35601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.125.keytab
--principal=kudu/127.10.114.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.125:33645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.114.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:32:11.831645 686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:32:11.833012 686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:32:11.844689 691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:11.844655 694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:11.844748 692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:32:11.845970 686 server_base.cc:1034] running on GCE node
I20260925 09:32:11.846763 686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:32:11.848151 686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:32:11.849396 686 hybrid_clock.cc:648] HybridClock initialized: now 1790328731849324 us; error 50 us; skew 500 ppm
I20260925 09:32:11.849732 686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:32:11 dist-test-slave-9pj8 krb5kdc[599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328731, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:32:11.852859 686 init.cc:377] Logged in from keytab as kudu/127.10.114.125@KRBTEST.COM (short username kudu)
I20260925 09:32:11.854403 686 webserver.cc:460] Webserver started at http://127.10.114.125:39179/ using document root <none> and password file <none>
I20260925 09:32:11.855087 686 fs_manager.cc:359] Metadata directory not provided
I20260925 09:32:11.855216 686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:32:11.855537 686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:32:11.858464 686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-1/data/instance:
uuid: "0bfcbd6b91eb44e28b1489f45e4228f9"
format_stamp: "Formatted at 2026-09-25 09:32:11 on dist-test-slave-9pj8"
server_key: "73760e1981d7ecadd59fb2421235a75a"
server_key_iv: "bdcbc4665ffdc1dcab2c3fdfb16f2ace"
server_key_version: "encryptionkey@0"
I20260925 09:32:11.859262 686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-1/wal/instance:
uuid: "0bfcbd6b91eb44e28b1489f45e4228f9"
format_stamp: "Formatted at 2026-09-25 09:32:11 on dist-test-slave-9pj8"
server_key: "73760e1981d7ecadd59fb2421235a75a"
server_key_iv: "bdcbc4665ffdc1dcab2c3fdfb16f2ace"
server_key_version: "encryptionkey@0"
I20260925 09:32:11.864192 686 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260925 09:32:11.867679 701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:32:11.869064 686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:32:11.869261 686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-1/wal
uuid: "0bfcbd6b91eb44e28b1489f45e4228f9"
format_stamp: "Formatted at 2026-09-25 09:32:11 on dist-test-slave-9pj8"
server_key: "73760e1981d7ecadd59fb2421235a75a"
server_key_iv: "bdcbc4665ffdc1dcab2c3fdfb16f2ace"
server_key_version: "encryptionkey@0"
I20260925 09:32:11.869510 686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:32:11.882354 686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:32:11.886426 686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:32:11.886720 686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:32:11.905728 686 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.125:33645
I20260925 09:32:11.905747 752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.125:33645 every 8 connection(s)
I20260925 09:32:11.907109 686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-1/data/info.pb
I20260925 09:32:11.908274 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 686
I20260925 09:32:11.908457 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-1/wal/instance
I20260925 09:32:11.908968 10697 external_mini_cluster.cc:1428] Setting key 595c2433abfdc687ffb59868381f8d70
I20260925 09:32:11.911760 753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:32:11.921082 753 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:32:11 dist-test-slave-9pj8 krb5kdc[599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328731, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.125@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.114.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.124@KRBTEST.COM" created.
I20260925 09:32:11.942662 682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:11.923681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54414 (local address 127.10.114.126:39341)
0925 09:32:11.923873 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:11.923877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:11.925203 (+ 1326us) server_negotiation.cc:407] Connection header received
0925 09:32:11.927083 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:11.927089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:11.927177 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:11.927294 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:11.928841 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:11.929972 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:11.931964 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:11.932560 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:11.932715 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:11.936573 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:11.936579 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:11.936583 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:11.936616 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:11.939430 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:11.940365 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:11.940370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:11.940373 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:11.940498 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:11.940940 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:11.940946 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:11.940954 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:11.941217 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:11.941382 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:11.942104 (+ 722us) server_negotiation.cc:299] Negotiation successful
0925 09:32:11.942332 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:32:11.943681 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:11.924301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39341 (local address 127.0.0.1:54414)
0925 09:32:11.925030 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:11.925076 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 09:32:11.926809 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:32:11.927525 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:32:11.927533 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:32:11.928100 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:32:11.928652 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:11.928669 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:11.930138 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:11.930142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:11.931705 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:11.931731 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:11.932808 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:11.932814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:11.933048 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:11.933781 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:32:11.933802 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:32:11.936322 (+ 2520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:32:11.939582 (+ 3260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:11.939592 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:11.939609 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:11.940233 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:11.940629 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:11.940635 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:11.940638 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:11.940817 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:11.941359 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:32:11.941367 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:32:11.941723 (+ 356us) client_negotiation.cc:769] Sending connection context
0925 09:32:11.941994 (+ 271us) client_negotiation.cc:240] Negotiation successful
0925 09:32:11.942517 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":313,"threads_started":1}
I20260925 09:32:11.945510 753 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:32:11 dist-test-slave-9pj8 krb5kdc[599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328731, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.125@KRBTEST.COM for kudu/127.10.114.125@KRBTEST.COM
I20260925 09:32:11.960927 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:11.946299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33645 (local address 127.0.0.1:45358)
0925 09:32:11.946484 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:11.946504 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:32:11.946630 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:32:11.947626 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:32:11.947638 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:32:11.948160 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:32:11.948381 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:11.948392 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:11.949751 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:11.949759 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:11.951272 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:11.951288 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:11.952169 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:11.952178 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:11.952393 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:11.952991 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:32:11.953016 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:32:11.955366 (+ 2350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:32:11.958464 (+ 3098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:11.958471 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:11.958477 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:11.958827 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:11.959205 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:11.959212 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:11.959218 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:11.959358 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:11.959978 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:32:11.959987 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:32:11.960117 (+ 130us) client_negotiation.cc:769] Sending connection context
0925 09:32:11.960270 (+ 153us) client_negotiation.cc:240] Negotiation successful
0925 09:32:11.960535 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260925 09:32:11.960968 757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:11.946452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45358 (local address 127.10.114.125:33645)
0925 09:32:11.947151 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:11.947161 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:11.947195 (+ 34us) server_negotiation.cc:407] Connection header received
0925 09:32:11.947261 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:11.947269 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:11.947466 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:11.947584 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:11.948556 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:11.949613 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:11.951406 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:11.952028 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:11.952118 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:11.955562 (+ 3444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:11.955572 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:11.955584 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:11.955627 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:11.958322 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:11.958956 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:11.958964 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:11.958968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:11.959075 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:11.959469 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:11.959474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:11.959478 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:11.959849 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:11.960051 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:11.960427 (+ 376us) server_negotiation.cc:299] Negotiation successful
0925 09:32:11.960620 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":357,"threads_started":1}
I20260925 09:32:11.963529 753 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:32:11.964718 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:11.964292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:57900)
0925 09:32:11.964461 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:11.964582 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260925 09:32:11.965055 705 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
I20260925 09:32:11.966637 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:11.966202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:57916)
0925 09:32:11.966414 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:11.966521 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260925 09:32:11.966858 753 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:35601: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.124.keytab.
Entry for principal kudu/127.10.114.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.124.keytab.
I20260925 09:32:11.973476 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.124:35601
--webserver_interface=127.10.114.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.124.keytab
--principal=kudu/127.10.114.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:33833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:39341,127.10.114.125:33645,127.10.114.124:35601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:32:12.016160 753 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } attempt: 1
I20260925 09:32:12.017455 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.016952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:57922)
0925 09:32:12.017166 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.017274 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260925 09:32:12.018482 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.018114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:57928)
0925 09:32:12.018287 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.018365 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260925 09:32:12.018711 753 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:35601: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
I20260925 09:32:12.086933 753 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } attempt: 2
I20260925 09:32:12.088284 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.087673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:57932)
0925 09:32:12.087931 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.088074 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260925 09:32:12.089493 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.089054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:57946)
0925 09:32:12.089225 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.089319 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260925 09:32:12.089798 753 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:35601: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
I20260925 09:32:12.119954 679 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } attempt: 5
Sep 25 09:32:12 dist-test-slave-9pj8 krb5kdc[599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328731, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for kudu/127.10.114.125@KRBTEST.COM
I20260925 09:32:12.134774 757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.120841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45360 (local address 127.10.114.125:33645)
0925 09:32:12.121050 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:12.121056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:12.121202 (+ 146us) server_negotiation.cc:407] Connection header received
0925 09:32:12.121434 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:12.121439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:12.121520 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:12.121670 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:12.122579 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.123245 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.124499 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.124912 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.125057 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:12.128941 (+ 3884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:12.128946 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:12.128955 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:12.128992 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:12.131721 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.132751 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.132756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.132759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.132881 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.133401 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.133404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.133406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.133700 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:12.133810 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:12.134374 (+ 564us) server_negotiation.cc:299] Negotiation successful
0925 09:32:12.134558 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 09:32:12.135095 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.120815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33645 (local address 127.0.0.1:45360)
0925 09:32:12.121091 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.121116 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 09:32:12.121282 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:32:12.121702 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:32:12.121706 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:32:12.122229 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:32:12.122400 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:12.122409 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.123384 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.123391 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:12.124346 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:12.124358 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.125068 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.125073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:12.125313 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:12.126063 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:32:12.126080 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:32:12.128653 (+ 2573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:32:12.132002 (+ 3349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:12.132010 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:12.132014 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:12.132616 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:12.133016 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:12.133024 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:12.133029 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:12.133260 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:12.133844 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:32:12.133852 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:32:12.134089 (+ 237us) client_negotiation.cc:769] Sending connection context
0925 09:32:12.134310 (+ 221us) client_negotiation.cc:240] Negotiation successful
0925 09:32:12.134668 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128}
I20260925 09:32:12.136438 679 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:32:12.137606 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.137131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:57954)
0925 09:32:12.137326 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.137428 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260925 09:32:12.138829 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.138297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:57958)
0925 09:32:12.138495 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.138617 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260925 09:32:12.139104 679 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:35601: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
I20260925 09:32:12.181036 753 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } attempt: 3
I20260925 09:32:12.182433 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.181905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:57970)
0925 09:32:12.182157 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.182270 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260925 09:32:12.183676 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.183210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:57986)
0925 09:32:12.183464 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.183554 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
W20260925 09:32:12.183918 753 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:35601: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
I20260925 09:32:12.190389 679 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } attempt: 1
I20260925 09:32:12.191620 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.191056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:57992)
0925 09:32:12.191280 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.191400 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260925 09:32:12.192781 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.192364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:58004)
0925 09:32:12.192524 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.192617 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260925 09:32:12.193141 679 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:35601: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
I20260925 09:32:12.261426 679 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } attempt: 2
I20260925 09:32:12.262732 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.262180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:58018)
0925 09:32:12.262472 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.262566 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260925 09:32:12.263964 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.263514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:58020)
0925 09:32:12.263692 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.263763 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260925 09:32:12.264289 679 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:35601: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
I20260925 09:32:12.327142 753 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } attempt: 4
I20260925 09:32:12.328450 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.327874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:58024)
0925 09:32:12.328132 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.328244 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260925 09:32:12.329526 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.329156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:58026)
0925 09:32:12.329308 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.329393 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 09:32:12.329774 753 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:35601: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
W20260925 09:32:12.330348 760 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 09:32:12.330641 760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:32:12.330710 760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:32:12.330792 760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:32:12.338120 760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:32:12.338212 760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:32:12.338269 760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:32:12.338311 760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:32:12.338377 760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:32:12.348492 760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:33833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:39341,127.10.114.125:33645,127.10.114.124:35601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.124.keytab
--principal=kudu/127.10.114.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.124:35601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.114.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:32:12.349741 760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:32:12.351004 760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:32:12.362811 765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:12.362777 766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:12.362864 768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:32:12.364125 760 server_base.cc:1034] running on GCE node
I20260925 09:32:12.364851 760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:32:12.366325 760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:32:12.367614 760 hybrid_clock.cc:648] HybridClock initialized: now 1790328732367559 us; error 33 us; skew 500 ppm
I20260925 09:32:12.368068 760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:32:12.370590 679 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } attempt: 3
Sep 25 09:32:12 dist-test-slave-9pj8 krb5kdc[599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328732, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:32:12.371891 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.371307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:58038)
0925 09:32:12.371578 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.371678 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260925 09:32:12.371910 760 init.cc:377] Logged in from keytab as kudu/127.10.114.124@KRBTEST.COM (short username kudu)
I20260925 09:32:12.373493 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.372754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:58040)
0925 09:32:12.373097 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.373301 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260925 09:32:12.373781 679 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:35601: Network error: Client connection negotiation failed: client connection to 127.10.114.124:35601: connect: Connection refused (error 111)
I20260925 09:32:12.374349 760 webserver.cc:460] Webserver started at http://127.10.114.124:41003/ using document root <none> and password file <none>
I20260925 09:32:12.375025 760 fs_manager.cc:359] Metadata directory not provided
I20260925 09:32:12.375121 760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:32:12.375479 760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:32:12.378558 760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-2/data/instance:
uuid: "802091fa4b35487ea7249d4ce211a78f"
format_stamp: "Formatted at 2026-09-25 09:32:12 on dist-test-slave-9pj8"
server_key: "92d34d5102749c03eaf09def5f2af61a"
server_key_iv: "d3dab62c35279443f6442b4458affee2"
server_key_version: "encryptionkey@0"
I20260925 09:32:12.379328 760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-2/wal/instance:
uuid: "802091fa4b35487ea7249d4ce211a78f"
format_stamp: "Formatted at 2026-09-25 09:32:12 on dist-test-slave-9pj8"
server_key: "92d34d5102749c03eaf09def5f2af61a"
server_key_iv: "d3dab62c35279443f6442b4458affee2"
server_key_version: "encryptionkey@0"
I20260925 09:32:12.384443 760 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260925 09:32:12.388017 775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:32:12.389405 760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260925 09:32:12.389577 760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-2/wal
uuid: "802091fa4b35487ea7249d4ce211a78f"
format_stamp: "Formatted at 2026-09-25 09:32:12 on dist-test-slave-9pj8"
server_key: "92d34d5102749c03eaf09def5f2af61a"
server_key_iv: "d3dab62c35279443f6442b4458affee2"
server_key_version: "encryptionkey@0"
I20260925 09:32:12.389798 760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:32:12.408407 760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:32:12.412725 760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:32:12.413031 760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:32:12.433135 760 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.124:35601
I20260925 09:32:12.433168 826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.124:35601 every 8 connection(s)
I20260925 09:32:12.434551 760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-2/data/info.pb
I20260925 09:32:12.437722 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 760
I20260925 09:32:12.437939 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790328385296020-10697-0/minicluster-data/master-2/wal/instance
I20260925 09:32:12.438355 10697 external_mini_cluster.cc:1428] Setting key b8f9677b285eb629c0dab7c57500dc30
I20260925 09:32:12.438423 827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:32:12.448041 827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:32:12 dist-test-slave-9pj8 krb5kdc[599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.124@KRBTEST.COM
Sep 25 09:32:12 dist-test-slave-9pj8 krb5kdc[599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328732, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.124@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:32:12.461715 830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.440271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58044 (local address 127.10.114.124:35601)
0925 09:32:12.441047 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:12.441059 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:12.441101 (+ 42us) server_negotiation.cc:407] Connection header received
0925 09:32:12.442999 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:12.443020 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:12.443383 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:12.443718 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:12.445508 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.446777 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.448533 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.449300 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.449447 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:12.454313 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:12.454320 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:12.454330 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:12.454365 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:12.457836 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.458504 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.458516 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.458521 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.458684 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.459134 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.459137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.459139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.459604 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:12.459948 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:12.460386 (+ 438us) server_negotiation.cc:299] Negotiation successful
0925 09:32:12.460799 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":361,"threads_started":1}
I20260925 09:32:12.466956 832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.451005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:39341 (local address 127.0.0.1:54428)
0925 09:32:12.451984 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.452025 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 09:32:12.452202 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:32:12.452674 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:32:12.452683 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:32:12.453550 (+ 867us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:32:12.453803 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:12.453812 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.454985 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.454989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:12.456792 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:12.456804 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.457566 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.457573 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:12.457764 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:12.458386 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:32:12.458407 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:32:12.460476 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:32:12.464036 (+ 3560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:12.464048 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:12.464054 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:12.464650 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:12.465111 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:12.465116 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:12.465120 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:12.465258 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:12.465842 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:32:12.465850 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:32:12.466037 (+ 187us) client_negotiation.cc:769] Sending connection context
0925 09:32:12.466220 (+ 183us) client_negotiation.cc:240] Negotiation successful
0925 09:32:12.466522 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"spinlock_wait_cycles":13056,"thread_start_us":548,"threads_started":1}
I20260925 09:32:12.466976 831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.451006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54428 (local address 127.10.114.126:39341)
0925 09:32:12.451534 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:12.451542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:12.452138 (+ 596us) server_negotiation.cc:407] Connection header received
0925 09:32:12.452349 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:12.452357 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:12.452452 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:12.452571 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:12.453975 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.454801 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.456948 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.457364 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.457519 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:12.460669 (+ 3150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:12.460674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:12.460677 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:12.460708 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:12.463900 (+ 3192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.464789 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.464794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.464797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.464971 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.465385 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.465390 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.465394 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.465679 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:12.465897 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:12.466341 (+ 444us) server_negotiation.cc:299] Negotiation successful
0925 09:32:12.466550 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":272,"threads_started":1}
I20260925 09:32:12.468760 827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:32:12 dist-test-slave-9pj8 krb5kdc[599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
Sep 25 09:32:12 dist-test-slave-9pj8 krb5kdc[599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328732, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.124@KRBTEST.COM for kudu/127.10.114.125@KRBTEST.COM
I20260925 09:32:12.480960 833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.464836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54444 (local address 127.10.114.126:39341)
0925 09:32:12.465450 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:12.465455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:12.465474 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:32:12.465511 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:12.465515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:12.465563 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:12.465692 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:12.466879 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.467930 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.469337 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.469958 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.470098 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:12.474997 (+ 4899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:12.475004 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:12.475009 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:12.475050 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:12.478066 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.478880 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.478884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.478886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.479003 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.479461 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.479467 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.479470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.479750 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:12.480078 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:12.480362 (+ 284us) server_negotiation.cc:299] Negotiation successful
0925 09:32:12.480582 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":291,"threads_started":1}
I20260925 09:32:12.483419 832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.469560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33645 (local address 127.0.0.1:45366)
0925 09:32:12.469710 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.469729 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:32:12.469813 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:32:12.470141 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:32:12.470146 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:32:12.470590 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:32:12.470764 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:12.470773 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.471754 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.471762 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:12.473254 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:12.473265 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.474310 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.474315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:12.474460 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:12.475011 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:32:12.475034 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:32:12.477497 (+ 2463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:32:12.480678 (+ 3181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:12.480684 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:12.480688 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:12.481192 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:12.481652 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:12.481656 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:12.481660 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:12.481860 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:12.482483 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:32:12.482487 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:32:12.482683 (+ 196us) client_negotiation.cc:769] Sending connection context
0925 09:32:12.482836 (+ 153us) client_negotiation.cc:240] Negotiation successful
0925 09:32:12.483126 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260925 09:32:12.483840 757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.469599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45366 (local address 127.10.114.125:33645)
0925 09:32:12.469780 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:12.469787 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:12.469806 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:32:12.469913 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:12.469918 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:12.469998 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:12.470094 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:12.470953 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.471606 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.473364 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.474166 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.474305 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:12.477697 (+ 3392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:12.477701 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:12.477705 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:12.477735 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:12.480545 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.481320 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.481325 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.481328 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.481504 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.481986 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.481990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.481991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.482348 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:12.482508 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:12.482924 (+ 416us) server_negotiation.cc:299] Negotiation successful
0925 09:32:12.483185 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 09:32:12.484879 827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:32:12 dist-test-slave-9pj8 krb5kdc[599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328732, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.124@KRBTEST.COM for kudu/127.10.114.124@KRBTEST.COM
Sep 25 09:32:12 dist-test-slave-9pj8 krb5kdc[599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.125@KRBTEST.COM
I20260925 09:32:12.498760 832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.485674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:58058)
0925 09:32:12.485811 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.485826 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:32:12.485900 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:32:12.486264 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:32:12.486268 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:32:12.486635 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:32:12.486760 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:12.486768 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.487695 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.487703 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:12.488587 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:12.488597 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.489296 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.489301 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:12.489498 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:12.490031 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:32:12.490046 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:32:12.492280 (+ 2234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:32:12.496096 (+ 3816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:12.496100 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:12.496103 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:12.496467 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:12.496898 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:12.496902 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:12.496904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:12.497037 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:12.497594 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:32:12.497598 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:32:12.497809 (+ 211us) client_negotiation.cc:769] Sending connection context
0925 09:32:12.498008 (+ 199us) client_negotiation.cc:240] Negotiation successful
0925 09:32:12.498328 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260925 09:32:12.499106 830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.485876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58058 (local address 127.10.114.124:35601)
0925 09:32:12.486054 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:12.486058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:12.486070 (+ 12us) server_negotiation.cc:407] Connection header received
0925 09:32:12.486100 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:12.486103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:12.486148 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:12.486250 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:12.486897 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.487561 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.488726 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.489133 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.489282 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:12.492485 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:12.492490 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:12.492494 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:12.492526 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:12.495924 (+ 3398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.496601 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.496607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.496613 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.496761 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.497158 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.497162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.497165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.497465 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:12.497572 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:12.498210 (+ 638us) server_negotiation.cc:299] Negotiation successful
0925 09:32:12.498583 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 09:32:12.501634 757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.482864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45380 (local address 127.10.114.125:33645)
0925 09:32:12.484086 (+ 1222us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:12.484096 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:12.484116 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:32:12.484169 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:12.484174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:12.484248 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:12.484354 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:12.486042 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.486755 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.487998 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.488427 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.488567 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:12.493499 (+ 4932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:12.493509 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:12.493516 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:12.493557 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:12.497048 (+ 3491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.497854 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.497861 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.497864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.498109 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.498640 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.498651 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.498657 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.499276 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:12.499585 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:12.500904 (+ 1319us) server_negotiation.cc:299] Negotiation successful
0925 09:32:12.501182 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1116,"thread_start_us":1029,"threads_started":1}
I20260925 09:32:12.505762 827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f: Bootstrap starting.
I20260925 09:32:12.508850 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:32:12.509796 827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f: Neither blocks nor log segments found. Creating new log.
I20260925 09:32:12.510830 827 log.cc:824] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f: Log is configured to *not* fsync() on all Append() calls
I20260925 09:32:12.513720 827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f: No bootstrap required, opened a new log
I20260925 09:32:12.511590 654 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d5b781c4737846ffa408dad0086a9f68"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54444
I20260925 09:32:12.520956 827 raft_consensus.cc:348] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b781c4737846ffa408dad0086a9f68" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } } peers { permanent_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } } peers { permanent_uuid: "802091fa4b35487ea7249d4ce211a78f" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } }
I20260925 09:32:12.521318 827 raft_consensus.cc:729] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 802091fa4b35487ea7249d4ce211a78f, State: Initialized, Role: FOLLOWER
I20260925 09:32:12.521763 827 consensus_queue.cc:260] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b781c4737846ffa408dad0086a9f68" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } } peers { permanent_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } } peers { permanent_uuid: "802091fa4b35487ea7249d4ce211a78f" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } }
I20260925 09:32:12.522806 836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b781c4737846ffa408dad0086a9f68" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } } peers { permanent_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } } peers { permanent_uuid: "802091fa4b35487ea7249d4ce211a78f" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } } }
I20260925 09:32:12.523202 836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:32:12.523949 827 sys_catalog.cc:564] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f [sys.catalog]: configured and running, proceeding with master startup.
W20260925 09:32:12.533500 847 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:32:12.533638 847 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:32:12.531270 760 master_runner.cc:186] Error getting master registration for 127.10.114.126:39341: OK, instance_id { permanent_uuid: "d5b781c4737846ffa408dad0086a9f68" instance_seqno: 1790328731407506 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 09:32:12.542541 760 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 09:32:12.551116 679 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } attempt: 4
Sep 25 09:32:12 dist-test-slave-9pj8 krb5kdc[599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328731, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for kudu/127.10.114.124@KRBTEST.COM
I20260925 09:32:12.573830 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.551822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:58064)
0925 09:32:12.552120 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.552144 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 09:32:12.552270 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:32:12.552671 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:32:12.552676 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:32:12.553187 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:32:12.553418 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:12.553432 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.554876 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.554883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:12.556365 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:12.556380 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.557474 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.557482 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:12.557695 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:12.564806 (+ 7111us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:32:12.564835 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:32:12.567605 (+ 2770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:32:12.571090 (+ 3485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:12.571096 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:12.571101 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:12.571489 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:12.571967 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:12.571972 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:12.571976 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:12.572133 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:12.572723 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:32:12.572728 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:32:12.572932 (+ 204us) client_negotiation.cc:769] Sending connection context
0925 09:32:12.573129 (+ 197us) client_negotiation.cc:240] Negotiation successful
0925 09:32:12.573424 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260925 09:32:12.574739 830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.551910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58064 (local address 127.10.114.124:35601)
0925 09:32:12.552161 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:12.552165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:12.552206 (+ 41us) server_negotiation.cc:407] Connection header received
0925 09:32:12.552409 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:12.552414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:12.552492 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:12.552613 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:12.553579 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.554716 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.556515 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.557316 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.557433 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:12.567868 (+ 10435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:12.567875 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:12.567880 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:12.567923 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:12.570934 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.571646 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.571652 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.571656 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.571841 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.572259 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.572264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.572267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.572584 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:12.572747 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:12.574191 (+ 1444us) server_negotiation.cc:299] Negotiation successful
0925 09:32:12.574423 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 09:32:12.579150 679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68: Bootstrap starting.
I20260925 09:32:12.582427 679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68: Neither blocks nor log segments found. Creating new log.
I20260925 09:32:12.583475 679 log.cc:824] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68: Log is configured to *not* fsync() on all Append() calls
I20260925 09:32:12.586380 679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68: No bootstrap required, opened a new log
I20260925 09:32:12.591485 679 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b781c4737846ffa408dad0086a9f68" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } } peers { permanent_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } } peers { permanent_uuid: "802091fa4b35487ea7249d4ce211a78f" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } }
I20260925 09:32:12.591856 679 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5b781c4737846ffa408dad0086a9f68, State: Initialized, Role: FOLLOWER
I20260925 09:32:12.592312 679 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b781c4737846ffa408dad0086a9f68" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } } peers { permanent_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } } peers { permanent_uuid: "802091fa4b35487ea7249d4ce211a78f" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } }
I20260925 09:32:12.593247 850 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b781c4737846ffa408dad0086a9f68" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } } peers { permanent_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } } peers { permanent_uuid: "802091fa4b35487ea7249d4ce211a78f" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } } }
I20260925 09:32:12.593678 850 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:32:12.594475 679 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:32:12.594614 654 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 09:32:12.594853 654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:32:12.598582 654 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b781c4737846ffa408dad0086a9f68" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } } peers { permanent_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } } peers { permanent_uuid: "802091fa4b35487ea7249d4ce211a78f" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } }
I20260925 09:32:12.601466 654 leader_election.cc:288] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [CANDIDATE]: Term 1 election: Requested vote from peers 0bfcbd6b91eb44e28b1489f45e4228f9 (127.10.114.125:33645), 802091fa4b35487ea7249d4ce211a78f (127.10.114.124:35601)
I20260925 09:32:12.603412 802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d5b781c4737846ffa408dad0086a9f68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "802091fa4b35487ea7249d4ce211a78f"
I20260925 09:32:12.603955 802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:32:12.605283 861 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:32:12.605476 861 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:32:12.612766 802 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5b781c4737846ffa408dad0086a9f68 in term 1.
I20260925 09:32:12.614270 631 leader_election.cc:302] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 802091fa4b35487ea7249d4ce211a78f, d5b781c4737846ffa408dad0086a9f68; no voters:
I20260925 09:32:12.614756 850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:32:12.609809 612 master_runner.cc:186] Error getting master registration for 127.10.114.125:33645: OK, instance_id { permanent_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9" instance_seqno: 1790328731900435 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 09:32:12.624667 612 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 09:32:12.628978 753 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } attempt: 5
I20260925 09:32:12.640801 850 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [term 1 LEADER]: Becoming Leader. State: Replica: d5b781c4737846ffa408dad0086a9f68, State: Running, Role: LEADER
I20260925 09:32:12.641701 850 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b781c4737846ffa408dad0086a9f68" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } } peers { permanent_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } } peers { permanent_uuid: "802091fa4b35487ea7249d4ce211a78f" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } }
W20260925 09:32:12.646653 633 tablet.cc:2369] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:32:12.669315 833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.616755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54450 (local address 127.10.114.126:39341)
0925 09:32:12.616929 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:12.616935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:12.620314 (+ 3379us) server_negotiation.cc:407] Connection header received
0925 09:32:12.628137 (+ 7823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:12.628143 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:12.628238 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:12.628330 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:12.630885 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.632395 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.638552 (+ 6157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.639497 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.639628 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:12.655432 (+ 15804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:12.655440 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:12.655454 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:12.655498 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:12.659579 (+ 4081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.664554 (+ 4975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.664562 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.664567 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.664743 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.667106 (+ 2363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.667112 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.667115 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.667566 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:12.667884 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:12.668634 (+ 750us) server_negotiation.cc:299] Negotiation successful
0925 09:32:12.668923 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260925 09:32:12.670949 781 tablet.cc:2369] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:32:12.673979 867 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5b781c4737846ffa408dad0086a9f68. Latest consensus state: current_term: 1 leader_uuid: "d5b781c4737846ffa408dad0086a9f68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b781c4737846ffa408dad0086a9f68" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } } peers { permanent_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } } peers { permanent_uuid: "802091fa4b35487ea7249d4ce211a78f" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } } }
I20260925 09:32:12.674736 867 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [sys.catalog]: This master's current role is: LEADER
Sep 25 09:32:12 dist-test-slave-9pj8 krb5kdc[599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328731, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.125@KRBTEST.COM for kudu/127.10.114.124@KRBTEST.COM
I20260925 09:32:12.684603 757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.621206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45394 (local address 127.10.114.125:33645)
0925 09:32:12.621366 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:12.621372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:12.635325 (+ 13953us) server_negotiation.cc:407] Connection header received
0925 09:32:12.635617 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:12.635625 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:12.635724 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:12.635825 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:12.643679 (+ 7854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.645132 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.650394 (+ 5262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.651050 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.651170 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:12.666720 (+ 15550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:12.666730 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:12.666734 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:12.666808 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:12.672278 (+ 5470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.679526 (+ 7248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.679532 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.679536 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.679691 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.682791 (+ 3100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.682797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.682800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.683170 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:12.683395 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:12.683958 (+ 563us) server_negotiation.cc:299] Negotiation successful
0925 09:32:12.684215 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 09:32:12.700742 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.629574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:35601 (local address 127.0.0.1:58086)
0925 09:32:12.629928 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:12.629960 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 09:32:12.630070 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:32:12.648508 (+ 18438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:32:12.648515 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:32:12.649088 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:32:12.649323 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:12.649335 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.655308 (+ 5973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.655316 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:12.656774 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:12.656791 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.675749 (+ 18958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.675755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:12.676016 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:12.676613 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:32:12.676636 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:32:12.685617 (+ 8981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:32:12.691429 (+ 5812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:12.691434 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:12.691437 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:12.691915 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:12.697562 (+ 5647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:12.697567 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:12.697571 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:12.697760 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:12.699699 (+ 1939us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:32:12.699705 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:32:12.699882 (+ 177us) client_negotiation.cc:769] Sending connection context
0925 09:32:12.700025 (+ 143us) client_negotiation.cc:240] Negotiation successful
0925 09:32:12.700337 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227}
I20260925 09:32:12.702610 830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.627989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58080 (local address 127.10.114.124:35601)
0925 09:32:12.628174 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:12.628179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:12.649326 (+ 21147us) server_negotiation.cc:407] Connection header received
0925 09:32:12.652036 (+ 2710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:12.652041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:12.652120 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:12.652183 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:12.670930 (+ 18747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.672226 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.674113 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.674734 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.674878 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:12.692132 (+ 17254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:12.692138 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:12.692143 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:12.692180 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:12.696105 (+ 3925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.696957 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.696962 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.696966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.697101 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.698188 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.698193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.698196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.698773 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:12.699042 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:12.701990 (+ 2948us) server_negotiation.cc:299] Negotiation successful
0925 09:32:12.702235 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 09:32:12.703567 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:12.630272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58086 (local address 127.10.114.124:35601)
0925 09:32:12.648164 (+ 17892us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:12.648170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:12.648192 (+ 22us) server_negotiation.cc:407] Connection header received
0925 09:32:12.648251 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:12.648256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:12.648333 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:12.648471 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:12.653962 (+ 5491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.655167 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.667964 (+ 12797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:12.675427 (+ 7463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:12.675556 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:12.685876 (+ 10320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:12.685884 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:12.685890 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:12.685930 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:12.689667 (+ 3737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.697216 (+ 7549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.697221 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.697226 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.697399 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:12.699105 (+ 1706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:12.699109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:12.699141 (+ 32us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:12.699499 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:12.699625 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:12.702893 (+ 3268us) server_negotiation.cc:299] Negotiation successful
0925 09:32:12.703112 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17742,"thread_start_us":17469,"threads_started":1}
I20260925 09:32:12.706737 873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:32:12.709107 873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:32:12.711695 753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9: Bootstrap starting.
I20260925 09:32:12.717404 753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9: Neither blocks nor log segments found. Creating new log.
I20260925 09:32:12.718760 753 log.cc:824] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9: Log is configured to *not* fsync() on all Append() calls
I20260925 09:32:12.723088 802 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f [term 1 FOLLOWER]: Refusing update from remote peer d5b781c4737846ffa408dad0086a9f68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:32:12.724252 867 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "802091fa4b35487ea7249d4ce211a78f" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:32:12.728232 753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9: No bootstrap required, opened a new log
I20260925 09:32:12.742925 836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5b781c4737846ffa408dad0086a9f68. Latest consensus state: current_term: 1 leader_uuid: "d5b781c4737846ffa408dad0086a9f68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b781c4737846ffa408dad0086a9f68" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } } peers { permanent_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } } peers { permanent_uuid: "802091fa4b35487ea7249d4ce211a78f" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } } }
I20260925 09:32:12.743280 836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:32:12.743820 753 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b781c4737846ffa408dad0086a9f68" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } } peers { permanent_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } } peers { permanent_uuid: "802091fa4b35487ea7249d4ce211a78f" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } }
I20260925 09:32:12.744239 753 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bfcbd6b91eb44e28b1489f45e4228f9, State: Initialized, Role: FOLLOWER
I20260925 09:32:12.744735 753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b781c4737846ffa408dad0086a9f68" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } } peers { permanent_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } } peers { permanent_uuid: "802091fa4b35487ea7249d4ce211a78f" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } }
I20260925 09:32:12.750531 874 mvcc.cc:205] Tried to move back new op lower bound from 7333186489218715648 to 7333186488910704640. Current Snapshot: MvccSnapshot[applied={T|T < 7333186489218715648}]
I20260925 09:32:12.780606 836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5b781c4737846ffa408dad0086a9f68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b781c4737846ffa408dad0086a9f68" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } } peers { permanent_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } } peers { permanent_uuid: "802091fa4b35487ea7249d4ce211a78f" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } } }
I20260925 09:32:12.780917 836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:32:12.788229 867 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5b781c4737846ffa408dad0086a9f68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b781c4737846ffa408dad0086a9f68" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } } peers { permanent_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } } peers { permanent_uuid: "802091fa4b35487ea7249d4ce211a78f" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } } }
I20260925 09:32:12.788565 867 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [sys.catalog]: This master's current role is: LEADER
I20260925 09:32:12.789254 878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b781c4737846ffa408dad0086a9f68" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } } peers { permanent_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } } peers { permanent_uuid: "802091fa4b35487ea7249d4ce211a78f" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } } }
I20260925 09:32:12.790690 878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:32:12.791326 753 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:32:12.791378 728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d5b781c4737846ffa408dad0086a9f68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9"
I20260925 09:32:12.791893 728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:32:12.797016 728 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5b781c4737846ffa408dad0086a9f68 in term 1.
I20260925 09:32:12.797881 727 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9 [term 1 FOLLOWER]: Refusing update from remote peer d5b781c4737846ffa408dad0086a9f68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:32:12.799050 873 catalog_manager.cc:1347] Generated new cluster ID: 14d74177c06449a69df6d93ffbdf73df
I20260925 09:32:12.799180 873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:32:12.804682 881 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:32:12.831673 873 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:32:12.833549 873 catalog_manager.cc:1504] Loading token signing keys...
W20260925 09:32:12.835618 892 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:32:12.835871 892 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:32:12.855741 873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68: Generated new TSK 0
I20260925 09:32:12.856511 881 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5b781c4737846ffa408dad0086a9f68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b781c4737846ffa408dad0086a9f68" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } } peers { permanent_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } } peers { permanent_uuid: "802091fa4b35487ea7249d4ce211a78f" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } } }
I20260925 09:32:12.856859 873 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:32:12.856932 881 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5b781c4737846ffa408dad0086a9f68 [sys.catalog]: This master's current role is: LEADER
I20260925 09:32:12.857298 878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5b781c4737846ffa408dad0086a9f68. Latest consensus state: current_term: 1 leader_uuid: "d5b781c4737846ffa408dad0086a9f68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b781c4737846ffa408dad0086a9f68" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } } peers { permanent_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } } peers { permanent_uuid: "802091fa4b35487ea7249d4ce211a78f" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } } }
I20260925 09:32:12.857602 878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:32:12.859184 896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5b781c4737846ffa408dad0086a9f68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b781c4737846ffa408dad0086a9f68" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 39341 } } peers { permanent_uuid: "0bfcbd6b91eb44e28b1489f45e4228f9" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33645 } } peers { permanent_uuid: "802091fa4b35487ea7249d4ce211a78f" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 35601 } } }
I20260925 09:32:12.861006 896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 09:32:13.038223 833 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:13.010277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54458 (local address 127.10.114.126:39341)
0925 09:32:13.010488 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:13.010494 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:13.034581 (+ 24087us) server_negotiation.cc:407] Connection header received
0925 09:32:13.036437 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:13.036443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:13.036553 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:13.036636 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:13.037907 (+ 1271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54458: BlockingRecv error: recv got EOF from 127.0.0.1:54458 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260925 09:32:13.042670 871 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:13.028227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58088 (local address 127.10.114.124:35601)
0925 09:32:13.028433 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:13.028439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:13.040318 (+ 11879us) server_negotiation.cc:407] Connection header received
0925 09:32:13.041069 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:13.041076 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:13.041183 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:13.041260 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:13.042404 (+ 1144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58088: BlockingRecv error: recv got EOF from 127.0.0.1:58088 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260925 09:32:13.044684 757 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:13.016610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45400 (local address 127.10.114.125:33645)
0925 09:32:13.016823 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:13.016831 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:13.039640 (+ 22809us) server_negotiation.cc:407] Connection header received
0925 09:32:13.043190 (+ 3550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:13.043197 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:13.043294 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:13.043366 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:13.044457 (+ 1091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45400: BlockingRecv error: recv got EOF from 127.0.0.1:45400 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260925 09:32:13.083966 757 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:13.075496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45410 (local address 127.10.114.125:33645)
0925 09:32:13.075663 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:13.075670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:13.081870 (+ 6200us) server_negotiation.cc:407] Connection header received
0925 09:32:13.082417 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:13.082423 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:13.082559 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:13.082644 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:13.083651 (+ 1007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45410: BlockingRecv error: recv got EOF from 127.0.0.1:45410 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260925 09:32:13.130906 757 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:13.121311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45426 (local address 127.10.114.125:33645)
0925 09:32:13.121526 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:13.121538 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:13.128921 (+ 7383us) server_negotiation.cc:407] Connection header received
0925 09:32:13.129414 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:13.129420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:13.129518 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:13.129592 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:13.130663 (+ 1071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45426: BlockingRecv error: recv got EOF from 127.0.0.1:45426 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260925 09:32:13.178277 757 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:13.168480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45434 (local address 127.10.114.125:33645)
0925 09:32:13.168677 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:13.168683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:13.176299 (+ 7616us) server_negotiation.cc:407] Connection header received
0925 09:32:13.176793 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:13.176799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:13.176910 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:13.177000 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:13.178028 (+ 1028us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45434: BlockingRecv error: recv got EOF from 127.0.0.1:45434 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260925 09:32:13.195441 757 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:13.193181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45446 (local address 127.10.114.125:33645)
0925 09:32:13.193422 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:13.193428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:13.193888 (+ 460us) server_negotiation.cc:407] Connection header received
0925 09:32:13.194066 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:13.194071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:13.194202 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:13.194324 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:13.195199 (+ 875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45446: BlockingRecv error: recv got EOF from 127.0.0.1:45446 (error 108)
Metrics: {"server-negotiator.queue_time_us":124}
W20260925 09:32:13.211989 757 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:13.209723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45454 (local address 127.10.114.125:33645)
0925 09:32:13.209899 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:13.209905 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:13.210402 (+ 497us) server_negotiation.cc:407] Connection header received
0925 09:32:13.210634 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:13.210640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:13.210728 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:13.210830 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:13.211654 (+ 824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45454: BlockingRecv error: recv got EOF from 127.0.0.1:45454 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260925 09:32:13.233428 757 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:13.231328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45470 (local address 127.10.114.125:33645)
0925 09:32:13.231545 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:13.231550 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:13.231929 (+ 379us) server_negotiation.cc:407] Connection header received
0925 09:32:13.232156 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:13.232160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:13.232263 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:13.232365 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:13.233204 (+ 839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45470: BlockingRecv error: recv got EOF from 127.0.0.1:45470 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260925 09:32:13.271472 757 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:13.269300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45482 (local address 127.10.114.125:33645)
0925 09:32:13.269531 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:13.269536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:13.269880 (+ 344us) server_negotiation.cc:407] Connection header received
0925 09:32:13.270086 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:13.270090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:13.270165 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:13.270260 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:13.271175 (+ 915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45482: BlockingRecv error: recv got EOF from 127.0.0.1:45482 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260925 09:32:13.341300 757 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:13.339186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45492 (local address 127.10.114.125:33645)
0925 09:32:13.339434 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:13.339441 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:13.339858 (+ 417us) server_negotiation.cc:407] Connection header received
0925 09:32:13.340100 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:13.340106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:13.340202 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:13.340348 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:13.341076 (+ 728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45492: BlockingRecv error: recv got EOF from 127.0.0.1:45492 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260925 09:32:13.474905 757 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:13.472945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38670 (local address 127.10.114.125:33645)
0925 09:32:13.473203 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:13.473209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:13.473472 (+ 263us) server_negotiation.cc:407] Connection header received
0925 09:32:13.473695 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:13.473699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:13.473768 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:13.473850 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:13.474674 (+ 824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38670: BlockingRecv error: recv got EOF from 127.0.0.1:38670 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
I20260925 09:32:13.535607 847 catalog_manager.cc:1259] Loaded cluster ID: 14d74177c06449a69df6d93ffbdf73df
I20260925 09:32:13.535773 847 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f: loading cluster ID for follower catalog manager: success
I20260925 09:32:13.538398 847 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f: acquiring CA information for follower catalog manager: success
I20260925 09:32:13.540361 847 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 802091fa4b35487ea7249d4ce211a78f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 09:32:13.739131 757 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:13.737322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38680 (local address 127.10.114.125:33645)
0925 09:32:13.737564 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:13.737570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:13.737856 (+ 286us) server_negotiation.cc:407] Connection header received
0925 09:32:13.737970 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:13.737974 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:13.738067 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:13.738161 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:13.738922 (+ 761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38680: BlockingRecv error: recv got EOF from 127.0.0.1:38680 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260925 09:32:13.838151 892 catalog_manager.cc:1259] Loaded cluster ID: 14d74177c06449a69df6d93ffbdf73df
I20260925 09:32:13.838263 892 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9: loading cluster ID for follower catalog manager: success
I20260925 09:32:13.841001 892 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9: acquiring CA information for follower catalog manager: success
I20260925 09:32:13.842885 892 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0bfcbd6b91eb44e28b1489f45e4228f9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 09:32:14.262146 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:14.256332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38694 (local address 127.10.114.125:33645)
0925 09:32:14.257001 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:14.257006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:14.257025 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:32:14.257192 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:14.257196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:14.257352 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:14.257462 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:32:14.257927 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:14.258604 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:14.259995 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:14.260410 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:14.260513 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:14.260990 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:32:14.261509 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:32:14.261619 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:14.261726 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 09:32:14.261795 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":388,"threads_started":1}
I20260925 09:32:14.278334 969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:14.272567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40414 (local address 127.10.114.124:35601)
0925 09:32:14.273077 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:14.273083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:14.273102 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:32:14.273282 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:14.273286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:14.273461 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:14.273659 (+ 198us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:32:14.274093 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:14.274787 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:14.276201 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:14.276601 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:14.276741 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:14.277226 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:32:14.277745 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:32:14.277913 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:14.277972 (+ 59us) server_negotiation.cc:299] Negotiation successful
0925 09:32:14.278049 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":254,"threads_started":1}
I20260925 09:32:14.303828 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:14.285296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38696 (local address 127.10.114.125:33645)
0925 09:32:14.285571 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:14.285580 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:14.297847 (+ 12267us) server_negotiation.cc:407] Connection header received
0925 09:32:14.298510 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:14.298516 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:14.298610 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:14.298708 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:32:14.299358 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:14.300228 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:14.301957 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:14.302367 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:14.302481 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:14.303061 (+ 580us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:32:14.303178 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:32:14.303272 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:14.303439 (+ 167us) server_negotiation.cc:299] Negotiation successful
0925 09:32:14.303521 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 09:32:14.304283 975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:14.284937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37958 (local address 127.10.114.126:39341)
0925 09:32:14.296354 (+ 11417us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:14.296363 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:14.297126 (+ 763us) server_negotiation.cc:407] Connection header received
0925 09:32:14.297455 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:14.297464 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:14.297762 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:14.297939 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:32:14.298561 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:14.299512 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:14.301660 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:14.302258 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:14.302390 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:14.302881 (+ 491us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:32:14.303324 (+ 443us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:32:14.303457 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:14.303896 (+ 439us) server_negotiation.cc:299] Negotiation successful
0925 09:32:14.304000 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11258,"thread_start_us":11444,"threads_started":1}
I20260925 09:32:14.304497 969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:14.285171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40416 (local address 127.10.114.124:35601)
0925 09:32:14.285986 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:14.285995 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:14.298083 (+ 12088us) server_negotiation.cc:407] Connection header received
0925 09:32:14.298343 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:14.298348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:14.298428 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:14.298535 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:32:14.299295 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:14.300329 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:14.302382 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:14.303028 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:14.303167 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:14.303589 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:32:14.303706 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:32:14.303787 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:14.304121 (+ 334us) server_negotiation.cc:299] Negotiation successful
0925 09:32:14.304202 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666}
I20260925 09:32:14.309297 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 612
I20260925 09:32:14.310654 674 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:32:14.421298 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 612
I20260925 09:32:14.442787 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 686
I20260925 09:32:14.443943 748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:32:14.561169 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 686
I20260925 09:32:14.582000 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 760
I20260925 09:32:14.583117 822 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:32:14.692610 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 760
2026-09-25T09:32:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5066 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:32:14 dist-test-slave-9pj8 krb5kdc[983](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:32:14 dist-test-slave-9pj8 krb5kdc[983](info): set up 2 sockets
Sep 25 09:32:14 dist-test-slave-9pj8 krb5kdc[983](info): commencing operation
krb5kdc: starting...
W20260925 09:32:15.796648 10697 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/test-user.keytab.
Sep 25 09:32:15 dist-test-slave-9pj8 krb5kdc[983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:32:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:32:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.114.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
Entry for principal kudu/127.10.114.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab.
I20260925 09:32:16.051452 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:42915
--webserver_interface=127.10.114.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:42915,127.10.114.125:33051,127.10.114.124:40347
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:32:16.373112 996 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 09:32:16.373412 996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:32:16.373517 996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:32:16.373663 996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:32:16.380738 996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:32:16.380808 996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:32:16.380929 996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:32:16.381011 996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:32:16.381098 996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:32:16.390161 996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:42915,127.10.114.125:33051,127.10.114.124:40347
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.126.keytab
--principal=kudu/127.10.114.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.126:42915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.114.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:32:16.391677 996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:32:16.393062 996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:32:16.404825 1001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:16.404906 1002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:16.404872 1004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:32:16.406095 996 server_base.cc:1034] running on GCE node
I20260925 09:32:16.406859 996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:32:16.408263 996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:32:16.409526 996 hybrid_clock.cc:648] HybridClock initialized: now 1790328736409475 us; error 60 us; skew 500 ppm
I20260925 09:32:16.409881 996 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:32:16 dist-test-slave-9pj8 krb5kdc[983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328736, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:32:16.412997 996 init.cc:377] Logged in from keytab as kudu/127.10.114.126@KRBTEST.COM (short username kudu)
I20260925 09:32:16.414327 996 webserver.cc:460] Webserver started at http://127.10.114.126:40545/ using document root <none> and password file <none>
I20260925 09:32:16.414928 996 fs_manager.cc:359] Metadata directory not provided
I20260925 09:32:16.415007 996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:32:16.415239 996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:32:16.418030 996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-0/data/instance:
uuid: "ac0833be9eaa4df4b0efd85baf930c61"
format_stamp: "Formatted at 2026-09-25 09:32:16 on dist-test-slave-9pj8"
server_key: "216a357301fc9fc5469c43dad0c92d08"
server_key_iv: "bb3995219ccb5e43b2758f18145a8d52"
server_key_version: "encryptionkey@0"
I20260925 09:32:16.418751 996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-0/wal/instance:
uuid: "ac0833be9eaa4df4b0efd85baf930c61"
format_stamp: "Formatted at 2026-09-25 09:32:16 on dist-test-slave-9pj8"
server_key: "216a357301fc9fc5469c43dad0c92d08"
server_key_iv: "bb3995219ccb5e43b2758f18145a8d52"
server_key_version: "encryptionkey@0"
I20260925 09:32:16.423674 996 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260925 09:32:16.426939 1011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:32:16.428346 996 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260925 09:32:16.428470 996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-0/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-0/wal
uuid: "ac0833be9eaa4df4b0efd85baf930c61"
format_stamp: "Formatted at 2026-09-25 09:32:16 on dist-test-slave-9pj8"
server_key: "216a357301fc9fc5469c43dad0c92d08"
server_key_iv: "bb3995219ccb5e43b2758f18145a8d52"
server_key_version: "encryptionkey@0"
I20260925 09:32:16.428623 996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:32:16.439692 996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:32:16.443516 996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:32:16.443713 996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:32:16.462450 996 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.126:42915
I20260925 09:32:16.462493 1062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.126:42915 every 8 connection(s)
I20260925 09:32:16.463800 996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-0/data/info.pb
I20260925 09:32:16.467278 1063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:32:16.473649 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 996
I20260925 09:32:16.473834 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-0/wal/instance
I20260925 09:32:16.473518 1063 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:32:16.474282 10697 external_mini_cluster.cc:1428] Setting key 0b401f592bd6b5ef6cb669f0fae30722
Sep 25 09:32:16 dist-test-slave-9pj8 krb5kdc[983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328736, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:32:16.497159 1066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:16.476694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38466 (local address 127.10.114.126:42915)
0925 09:32:16.478289 (+ 1595us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:16.478299 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:16.478347 (+ 48us) server_negotiation.cc:407] Connection header received
0925 09:32:16.480312 (+ 1965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:16.480322 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:16.480643 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:16.480853 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:16.482094 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:16.483968 (+ 1874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:16.485739 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:16.486300 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:16.486508 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:16.489895 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:16.489903 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:16.489914 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:16.489952 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:16.492899 (+ 2947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:16.493550 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:16.493560 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:16.493565 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:16.493706 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:16.494086 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:16.494093 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:16.494097 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:16.494562 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:16.494788 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:16.495298 (+ 510us) server_negotiation.cc:299] Negotiation successful
0925 09:32:16.495659 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":1375,"thread_start_us":1180,"threads_started":1}
I20260925 09:32:16.497120 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:16.476709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42915 (local address 127.0.0.1:38466)
0925 09:32:16.477712 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:16.477785 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0925 09:32:16.479922 (+ 2137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:32:16.480890 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:32:16.480895 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:32:16.481490 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:32:16.481871 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:16.481888 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:16.484128 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:16.484137 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:16.485587 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:16.485599 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:16.486548 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:16.486554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:16.486758 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:16.487431 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:32:16.487459 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:32:16.489677 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:32:16.493041 (+ 3364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:16.493053 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:16.493064 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:16.493421 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:16.493818 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:16.493824 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:16.493827 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:16.493981 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:16.494702 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:32:16.494709 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:32:16.494935 (+ 226us) client_negotiation.cc:769] Sending connection context
0925 09:32:16.495181 (+ 246us) client_negotiation.cc:240] Negotiation successful
0925 09:32:16.495788 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":168,"thread_start_us":389,"threads_started":1}
WARNING: no policy specified for kudu/127.10.114.125@KRBTEST.COM; defaulting to no policy
I20260925 09:32:16.500785 1063 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.125@KRBTEST.COM" created.
I20260925 09:32:16.503396 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:16.501608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33051 (local address 127.0.0.1:50354)
0925 09:32:16.503039 (+ 1431us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:16.503209 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1240}
W20260925 09:32:16.503700 1013 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33051: connect: Connection refused (error 111)
I20260925 09:32:16.505666 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:16.505260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33051 (local address 127.0.0.1:50362)
0925 09:32:16.505405 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:16.505541 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260925 09:32:16.505919 1063 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.125:33051: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33051: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.125.keytab.
Entry for principal kudu/127.10.114.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.125.keytab.
I20260925 09:32:16.539314 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.125:33051
--webserver_interface=127.10.114.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.125.keytab
--principal=kudu/127.10.114.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:42915,127.10.114.125:33051,127.10.114.124:40347
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:32:16.555236 1063 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } attempt: 1
I20260925 09:32:16.556557 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:16.556045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33051 (local address 127.0.0.1:50366)
0925 09:32:16.556289 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:16.556415 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260925 09:32:16.557581 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:16.557232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33051 (local address 127.0.0.1:50382)
0925 09:32:16.557385 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:16.557464 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260925 09:32:16.557777 1063 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.125:33051: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33051: connect: Connection refused (error 111)
I20260925 09:32:16.626022 1063 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } attempt: 2
I20260925 09:32:16.627241 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:16.626738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33051 (local address 127.0.0.1:50388)
0925 09:32:16.627012 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:16.627103 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260925 09:32:16.628317 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:16.627955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33051 (local address 127.0.0.1:50404)
0925 09:32:16.628115 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:16.628188 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 09:32:16.628558 1063 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.125:33051: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33051: connect: Connection refused (error 111)
I20260925 09:32:16.719893 1063 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } attempt: 3
I20260925 09:32:16.721032 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:16.720583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33051 (local address 127.0.0.1:50420)
0925 09:32:16.720790 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:16.720884 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260925 09:32:16.722002 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:16.721646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33051 (local address 127.0.0.1:50426)
0925 09:32:16.721790 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:16.721862 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260925 09:32:16.722164 1063 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.125:33051: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33051: connect: Connection refused (error 111)
I20260925 09:32:16.865417 1063 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } attempt: 4
I20260925 09:32:16.866748 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:16.866133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33051 (local address 127.0.0.1:50442)
0925 09:32:16.866494 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:16.866593 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201}
I20260925 09:32:16.867944 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:16.867495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33051 (local address 127.0.0.1:50446)
0925 09:32:16.867652 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:16.867748 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260925 09:32:16.868173 1063 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.125:33051: Network error: Client connection negotiation failed: client connection to 127.10.114.125:33051: connect: Connection refused (error 111)
W20260925 09:32:16.870570 1070 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 09:32:16.870901 1070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:32:16.871050 1070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:32:16.871167 1070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:32:16.878481 1070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:32:16.878561 1070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:32:16.878666 1070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:32:16.878743 1070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:32:16.878854 1070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:32:16.890591 1070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:42915,127.10.114.125:33051,127.10.114.124:40347
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.125.keytab
--principal=kudu/127.10.114.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.125:33051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.114.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:32:16.892329 1070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:32:16.893973 1070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:32:16.906988 1076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:16.907013 1075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:16.907636 1078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:32:16.908351 1070 server_base.cc:1034] running on GCE node
I20260925 09:32:16.909137 1070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:32:16.910549 1070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:32:16.911893 1070 hybrid_clock.cc:648] HybridClock initialized: now 1790328736911755 us; error 51 us; skew 500 ppm
I20260925 09:32:16.912250 1070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:32:16 dist-test-slave-9pj8 krb5kdc[983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328736, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:32:16.915439 1070 init.cc:377] Logged in from keytab as kudu/127.10.114.125@KRBTEST.COM (short username kudu)
I20260925 09:32:16.917052 1070 webserver.cc:460] Webserver started at http://127.10.114.125:43433/ using document root <none> and password file <none>
I20260925 09:32:16.917773 1070 fs_manager.cc:359] Metadata directory not provided
I20260925 09:32:16.917929 1070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:32:16.918255 1070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:32:16.921252 1070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-1/data/instance:
uuid: "d3c6975853704da6bd1bead42ffe6078"
format_stamp: "Formatted at 2026-09-25 09:32:16 on dist-test-slave-9pj8"
server_key: "8c20b9537b9614abd6a8d6ba40de031b"
server_key_iv: "a96e0766b1d299aba747ff200d2f9f61"
server_key_version: "encryptionkey@0"
I20260925 09:32:16.922009 1070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-1/wal/instance:
uuid: "d3c6975853704da6bd1bead42ffe6078"
format_stamp: "Formatted at 2026-09-25 09:32:16 on dist-test-slave-9pj8"
server_key: "8c20b9537b9614abd6a8d6ba40de031b"
server_key_iv: "a96e0766b1d299aba747ff200d2f9f61"
server_key_version: "encryptionkey@0"
I20260925 09:32:16.927234 1070 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:32:16.930655 1085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:32:16.932289 1070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260925 09:32:16.932466 1070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-1/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-1/wal
uuid: "d3c6975853704da6bd1bead42ffe6078"
format_stamp: "Formatted at 2026-09-25 09:32:16 on dist-test-slave-9pj8"
server_key: "8c20b9537b9614abd6a8d6ba40de031b"
server_key_iv: "a96e0766b1d299aba747ff200d2f9f61"
server_key_version: "encryptionkey@0"
I20260925 09:32:16.932719 1070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:32:16.966459 1070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:32:16.971197 1070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:32:16.971499 1070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:32:16.990033 1070 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.125:33051
I20260925 09:32:16.990052 1136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.125:33051 every 8 connection(s)
I20260925 09:32:16.991441 1070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-1/data/info.pb
I20260925 09:32:16.995062 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 1070
I20260925 09:32:16.995155 1137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:32:16.995285 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-1/wal/instance
I20260925 09:32:16.995712 10697 external_mini_cluster.cc:1428] Setting key a60a937951bc3e81fc82fc906af42931
I20260925 09:32:17.001744 1137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:32:17 dist-test-slave-9pj8 krb5kdc[983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328736, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.125@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.114.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.114.124@KRBTEST.COM" created.
I20260925 09:32:17.024971 1140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.004249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38472 (local address 127.10.114.126:42915)
0925 09:32:17.004782 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.004788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.005762 (+ 974us) server_negotiation.cc:407] Connection header received
0925 09:32:17.007739 (+ 1977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.007748 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.007887 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.008021 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.009545 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.010722 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.012413 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.013147 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.013295 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.017270 (+ 3975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:17.017277 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:17.017283 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:17.017334 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:17.020928 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.021935 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.021940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.021942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.022138 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.022702 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.022708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.022712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.023114 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:17.023293 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:17.024156 (+ 863us) server_negotiation.cc:299] Negotiation successful
0925 09:32:17.024431 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":272,"threads_started":1}
I20260925 09:32:17.025627 1141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.004809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42915 (local address 127.0.0.1:38472)
0925 09:32:17.005580 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.005625 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 09:32:17.007477 (+ 1852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:32:17.008237 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:32:17.008245 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:32:17.008795 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:32:17.009326 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:17.009343 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.010909 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.010913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:17.012259 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:17.012272 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.013375 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.013380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:17.013695 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.014433 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:32:17.014478 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:32:17.016977 (+ 2499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:32:17.021208 (+ 4231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:17.021217 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:17.021233 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:17.021755 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:17.022273 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:17.022277 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:17.022280 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:17.022541 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:17.023279 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:32:17.023285 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:32:17.023681 (+ 396us) client_negotiation.cc:769] Sending connection context
0925 09:32:17.024017 (+ 336us) client_negotiation.cc:240] Negotiation successful
0925 09:32:17.024458 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":324,"threads_started":1}
I20260925 09:32:17.027522 1137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:32:17 dist-test-slave-9pj8 krb5kdc[983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328736, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.125@KRBTEST.COM for kudu/127.10.114.125@KRBTEST.COM
I20260925 09:32:17.044576 1141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.028528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33051 (local address 127.0.0.1:50460)
0925 09:32:17.028727 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.028751 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 09:32:17.028866 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:32:17.030001 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:32:17.030005 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:32:17.030362 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:32:17.030531 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:17.030539 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.032723 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.032732 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:17.033519 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:17.033529 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.034765 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.034769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:17.034934 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.035561 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:32:17.035577 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:32:17.037734 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:32:17.042058 (+ 4324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:17.042063 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:17.042065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:17.042423 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:17.042892 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:17.042895 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:17.042898 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:17.043040 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:17.043866 (+ 826us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:32:17.043869 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:32:17.043991 (+ 122us) client_negotiation.cc:769] Sending connection context
0925 09:32:17.044107 (+ 116us) client_negotiation.cc:240] Negotiation successful
0925 09:32:17.044300 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":3}
I20260925 09:32:17.045097 1142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.028669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50460 (local address 127.10.114.125:33051)
0925 09:32:17.029484 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.029494 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.029527 (+ 33us) server_negotiation.cc:407] Connection header received
0925 09:32:17.029586 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.029597 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.029838 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.029992 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.030726 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.032553 (+ 1827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.033659 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.034611 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.034749 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.037947 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:17.037961 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:17.037981 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:17.038026 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:17.041921 (+ 3895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.042554 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.042564 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.042571 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.042721 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.043162 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.043169 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.043175 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.043673 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:17.043895 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:17.044289 (+ 394us) server_negotiation.cc:299] Negotiation successful
0925 09:32:17.044561 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":595,"spinlock_wait_cycles":21760,"thread_start_us":431,"threads_started":1}
I20260925 09:32:17.047616 1137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:32:17.048730 1141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.048293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41782)
0925 09:32:17.048494 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.048593 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260925 09:32:17.048965 1087 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
I20260925 09:32:17.050489 1141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.050147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41796)
0925 09:32:17.050315 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.050399 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260925 09:32:17.050701 1137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:40347: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.114.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.124.keytab.
Entry for principal kudu/127.10.114.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.124.keytab.
I20260925 09:32:17.059247 10697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
/tmp/dist-test-taskMvTxiP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.124:40347
--webserver_interface=127.10.114.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.124.keytab
--principal=kudu/127.10.114.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.114.84:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.114.126:42915,127.10.114.125:33051,127.10.114.124:40347
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:32:17.099996 1137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } attempt: 1
I20260925 09:32:17.101354 1141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.100798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41806)
0925 09:32:17.101032 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.101158 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260925 09:32:17.103422 1141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.103041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41808)
0925 09:32:17.103224 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.103312 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260925 09:32:17.103675 1137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:40347: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
I20260925 09:32:17.167467 1063 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } attempt: 5
I20260925 09:32:17.171934 1137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } attempt: 2
I20260925 09:32:17.173040 1141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.172515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41818)
0925 09:32:17.172729 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.172837 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260925 09:32:17.174340 1141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.173922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41830)
0925 09:32:17.174096 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.174186 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260925 09:32:17.174646 1137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:40347: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
Sep 25 09:32:17 dist-test-slave-9pj8 krb5kdc[983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328736, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for kudu/127.10.114.125@KRBTEST.COM
I20260925 09:32:17.182063 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.168337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33051 (local address 127.0.0.1:50466)
0925 09:32:17.168564 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.168586 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:32:17.168727 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:32:17.169122 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:32:17.169127 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:32:17.169578 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:32:17.169757 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:17.169766 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.170777 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.170786 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:17.171676 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:17.171688 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.172695 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.172701 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:17.172863 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.173568 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:32:17.173591 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:32:17.176033 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:32:17.179678 (+ 3645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:17.179681 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:17.179684 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:17.180154 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:17.180560 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:17.180564 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:17.180566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:17.180706 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:17.181308 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:32:17.181311 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:32:17.181496 (+ 185us) client_negotiation.cc:769] Sending connection context
0925 09:32:17.181608 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 09:32:17.181828 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260925 09:32:17.182263 1142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.168513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50466 (local address 127.10.114.125:33051)
0925 09:32:17.168711 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.168717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.168739 (+ 22us) server_negotiation.cc:407] Connection header received
0925 09:32:17.168887 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.168892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.168970 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.169086 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.169923 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.170632 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.171843 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.172517 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.173211 (+ 694us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.176229 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:17.176235 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:17.176240 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:17.176281 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:17.179525 (+ 3244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.180273 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.180279 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.180282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.180438 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.180826 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.180830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.180833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.181186 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:17.181356 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:17.181763 (+ 407us) server_negotiation.cc:299] Negotiation successful
0925 09:32:17.181988 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 09:32:17.183247 1063 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:32:17.184278 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.183897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41834)
0925 09:32:17.184082 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.184155 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260925 09:32:17.185345 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.184933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41838)
0925 09:32:17.185092 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.185189 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260925 09:32:17.185618 1063 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:40347: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
I20260925 09:32:17.236884 1063 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } attempt: 1
I20260925 09:32:17.238270 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.237738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41846)
0925 09:32:17.237992 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.238086 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260925 09:32:17.239463 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.239074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41856)
0925 09:32:17.239224 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.239309 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260925 09:32:17.239715 1063 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:40347: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
I20260925 09:32:17.265911 1137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } attempt: 3
I20260925 09:32:17.267184 1141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.266665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41870)
0925 09:32:17.266893 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.267006 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260925 09:32:17.268404 1141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.267951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41876)
0925 09:32:17.268186 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.268277 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
W20260925 09:32:17.268647 1137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:40347: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
I20260925 09:32:17.308000 1063 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } attempt: 2
I20260925 09:32:17.309257 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.308737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41884)
0925 09:32:17.309011 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.309105 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260925 09:32:17.310287 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.309944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41894)
0925 09:32:17.310093 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.310172 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260925 09:32:17.310524 1063 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:40347: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
W20260925 09:32:17.402635 1145 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 09:32:17.403013 1145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:32:17.403153 1145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:32:17.403299 1145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:32:17.410605 1145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:32:17.410729 1145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:32:17.410835 1145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:32:17.410923 1145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:32:17.411008 1145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:32:17.411953 1137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } attempt: 4
I20260925 09:32:17.413262 1141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.412719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41902)
0925 09:32:17.412977 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.413094 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260925 09:32:17.414458 1141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.414123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41910)
0925 09:32:17.414276 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.414354 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260925 09:32:17.414753 1137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:40347: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
I20260925 09:32:17.416746 1063 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } attempt: 3
I20260925 09:32:17.417773 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.417286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41922)
0925 09:32:17.417489 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.417589 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260925 09:32:17.418802 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.418414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41932)
0925 09:32:17.418594 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.418681 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260925 09:32:17.419062 1063 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.114.124:40347: Network error: Client connection negotiation failed: client connection to 127.10.114.124:40347: connect: Connection refused (error 111)
I20260925 09:32:17.422124 1145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.114.84:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.114.126:42915,127.10.114.125:33051,127.10.114.124:40347
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/krb5kdc/kudu_127.10.114.124.keytab
--principal=kudu/127.10.114.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.114.124:40347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.114.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:32:17.423431 1145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:32:17.424896 1145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:32:17.437536 1150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:17.437597 1151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:32:17.437623 1153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:32:17.438725 1145 server_base.cc:1034] running on GCE node
I20260925 09:32:17.439581 1145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:32:17.441015 1145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:32:17.442235 1145 hybrid_clock.cc:648] HybridClock initialized: now 1790328737442184 us; error 35 us; skew 500 ppm
I20260925 09:32:17.442600 1145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:32:17 dist-test-slave-9pj8 krb5kdc[983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328737, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:32:17.445926 1145 init.cc:377] Logged in from keytab as kudu/127.10.114.124@KRBTEST.COM (short username kudu)
I20260925 09:32:17.447460 1145 webserver.cc:460] Webserver started at http://127.10.114.124:37651/ using document root <none> and password file <none>
I20260925 09:32:17.448216 1145 fs_manager.cc:359] Metadata directory not provided
I20260925 09:32:17.448302 1145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:32:17.448568 1145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:32:17.451395 1145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-2/data/instance:
uuid: "f254a73073dc4eb68845b04632a7ff8b"
format_stamp: "Formatted at 2026-09-25 09:32:17 on dist-test-slave-9pj8"
server_key: "b72b54138a3e38cd60375c5098fcf811"
server_key_iv: "61d13f81b517387896cf3956e427cd7f"
server_key_version: "encryptionkey@0"
I20260925 09:32:17.452235 1145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-2/wal/instance:
uuid: "f254a73073dc4eb68845b04632a7ff8b"
format_stamp: "Formatted at 2026-09-25 09:32:17 on dist-test-slave-9pj8"
server_key: "b72b54138a3e38cd60375c5098fcf811"
server_key_iv: "61d13f81b517387896cf3956e427cd7f"
server_key_version: "encryptionkey@0"
I20260925 09:32:17.457104 1145 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 09:32:17.460337 1160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:32:17.461777 1145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:32:17.461954 1145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-2/data,/tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-2/wal
uuid: "f254a73073dc4eb68845b04632a7ff8b"
format_stamp: "Formatted at 2026-09-25 09:32:17 on dist-test-slave-9pj8"
server_key: "b72b54138a3e38cd60375c5098fcf811"
server_key_iv: "61d13f81b517387896cf3956e427cd7f"
server_key_version: "encryptionkey@0"
I20260925 09:32:17.462186 1145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:32:17.511034 1145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:32:17.515499 1145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:32:17.515708 1145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:32:17.534216 1145 rpc_server.cc:307] RPC server started. Bound to: 127.10.114.124:40347
I20260925 09:32:17.534240 1211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.114.124:40347 every 8 connection(s)
I20260925 09:32:17.535588 1145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-2/data/info.pb
I20260925 09:32:17.539513 1212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:32:17.544795 10697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu as pid 1145
I20260925 09:32:17.544989 10697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMvTxiP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790328385296020-10697-0/minicluster-data/master-2/wal/instance
I20260925 09:32:17.545418 10697 external_mini_cluster.cc:1428] Setting key 9d017e39a01412e74a1d767ab2d6d23b
I20260925 09:32:17.546167 1212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:32:17 dist-test-slave-9pj8 krb5kdc[983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328737, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.124@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
Sep 25 09:32:17 dist-test-slave-9pj8 krb5kdc[983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.124@KRBTEST.COM
I20260925 09:32:17.567034 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.549078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38486 (local address 127.10.114.126:42915)
0925 09:32:17.549552 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.549558 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.550135 (+ 577us) server_negotiation.cc:407] Connection header received
0925 09:32:17.550570 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.550578 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.550668 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.550780 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.552491 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.553558 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.555524 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.556204 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.556339 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.560485 (+ 4146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:17.560489 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:17.560493 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:17.560524 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:17.563381 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.564172 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.564179 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.564183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.564341 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.564823 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.564828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.564831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.565199 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:17.565319 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:17.566258 (+ 939us) server_negotiation.cc:299] Negotiation successful
0925 09:32:17.566539 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":223,"threads_started":1}
I20260925 09:32:17.568044 1215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.547334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41942 (local address 127.10.114.124:40347)
0925 09:32:17.548185 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.548195 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.548232 (+ 37us) server_negotiation.cc:407] Connection header received
0925 09:32:17.550313 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.550333 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.550619 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.550815 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.552541 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.554026 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.555712 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.556598 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.556693 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.561106 (+ 4413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:17.561113 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:17.561123 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:17.561147 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:17.563717 (+ 2570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.564495 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.564503 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.564506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.564679 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.565108 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.565111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.565113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.565781 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:17.566094 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:17.566359 (+ 265us) server_negotiation.cc:299] Negotiation successful
0925 09:32:17.566721 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":176,"server-negotiator.queue_time_us":643,"thread_start_us":416,"threads_started":1}
I20260925 09:32:17.568092 1217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.549140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.126:42915 (local address 127.0.0.1:38486)
0925 09:32:17.549986 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.550028 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 09:32:17.550343 (+ 315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:32:17.550870 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:32:17.550879 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:32:17.551740 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:32:17.552284 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:17.552313 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.553713 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.553717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:17.555379 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:17.555394 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.556444 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.556449 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:17.556638 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.557412 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:32:17.557444 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:32:17.560297 (+ 2853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:32:17.563513 (+ 3216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:17.563525 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:17.563541 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:17.564046 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:17.564470 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:17.564475 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:17.564478 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:17.564698 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:17.565320 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:32:17.565328 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:32:17.565800 (+ 472us) client_negotiation.cc:769] Sending connection context
0925 09:32:17.566100 (+ 300us) client_negotiation.cc:240] Negotiation successful
0925 09:32:17.566602 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":71,"spinlock_wait_cycles":15616,"thread_start_us":405,"threads_started":1}
I20260925 09:32:17.570622 1212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:32:17 dist-test-slave-9pj8 krb5kdc[983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328737, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.124@KRBTEST.COM for kudu/127.10.114.125@KRBTEST.COM
Sep 25 09:32:17 dist-test-slave-9pj8 krb5kdc[983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.126@KRBTEST.COM
I20260925 09:32:17.589680 1217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.571487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.125:33051 (local address 127.0.0.1:50468)
0925 09:32:17.571675 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.571693 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:32:17.571832 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:32:17.573282 (+ 1450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:32:17.573288 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:32:17.573762 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:32:17.573941 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:17.573951 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.574905 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.574913 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:17.575844 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:17.575858 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.576559 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.576564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:17.576755 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.582188 (+ 5433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:32:17.582205 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:32:17.584327 (+ 2122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:32:17.587223 (+ 2896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:17.587229 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:17.587233 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:17.587582 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:17.588023 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:17.588028 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:17.588032 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:17.588186 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:17.588697 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:32:17.588702 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:32:17.588851 (+ 149us) client_negotiation.cc:769] Sending connection context
0925 09:32:17.588991 (+ 140us) client_negotiation.cc:240] Negotiation successful
0925 09:32:17.589311 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260925 09:32:17.589707 1142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.571490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50468 (local address 127.10.114.125:33051)
0925 09:32:17.571667 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.571671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.571741 (+ 70us) server_negotiation.cc:407] Connection header received
0925 09:32:17.571941 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.571945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.572007 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.572109 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.574102 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.574761 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.576002 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.576400 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.576550 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.584523 (+ 7973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:17.584527 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:17.584530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:17.584561 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:17.587084 (+ 2523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.587710 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.587720 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.587723 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.587905 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.588301 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.588305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.588309 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.588565 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:17.588734 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:17.589109 (+ 375us) server_negotiation.cc:299] Negotiation successful
0925 09:32:17.589327 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 09:32:17.590699 1212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:32:17.591246 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.571083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38492 (local address 127.10.114.126:42915)
0925 09:32:17.571849 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.571854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.571872 (+ 18us) server_negotiation.cc:407] Connection header received
0925 09:32:17.571923 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.571930 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.572012 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.572116 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.573222 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.574275 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.575546 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.576008 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.576150 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.585647 (+ 9497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:17.585651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:17.585654 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:17.585685 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:17.588463 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.589168 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.589174 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.589177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.589311 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.589695 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.589701 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.589704 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.590085 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:17.590339 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:17.590727 (+ 388us) server_negotiation.cc:299] Negotiation successful
0925 09:32:17.590946 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612}
I20260925 09:32:17.596264 1063 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } attempt: 4
Sep 25 09:32:17 dist-test-slave-9pj8 krb5kdc[983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328737, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.124@KRBTEST.COM for kudu/127.10.114.124@KRBTEST.COM
Sep 25 09:32:17 dist-test-slave-9pj8 krb5kdc[983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.114.125@KRBTEST.COM
I20260925 09:32:17.605511 1217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.591437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41944)
0925 09:32:17.591604 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.591624 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:32:17.591719 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:32:17.592148 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:32:17.592152 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:32:17.592491 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:32:17.592611 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:17.592618 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.593918 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.593922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:17.594952 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:17.594965 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.595657 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.595661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:17.595909 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.596505 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:32:17.596531 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:32:17.599547 (+ 3016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:32:17.602757 (+ 3210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:17.602765 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:17.602768 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:17.603274 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:17.603672 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:17.603680 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:17.603684 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:17.603866 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:17.604627 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:32:17.604632 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:32:17.604793 (+ 161us) client_negotiation.cc:769] Sending connection context
0925 09:32:17.604927 (+ 134us) client_negotiation.cc:240] Negotiation successful
0925 09:32:17.605236 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":2}
Sep 25 09:32:17 dist-test-slave-9pj8 krb5kdc[983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328736, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.126@KRBTEST.COM for kudu/127.10.114.124@KRBTEST.COM
I20260925 09:32:17.606240 1215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.591764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41944 (local address 127.10.114.124:40347)
0925 09:32:17.591928 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.591933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.591944 (+ 11us) server_negotiation.cc:407] Connection header received
0925 09:32:17.591975 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.591978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.592025 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.592128 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.592736 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.593788 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.595098 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.595521 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.595656 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.599743 (+ 4087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:17.599749 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:17.599755 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:17.599837 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:17.602627 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.603403 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.603409 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.603412 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.603546 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.604169 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.604175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.604180 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.604496 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:17.604650 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:17.605854 (+ 1204us) server_negotiation.cc:299] Negotiation successful
0925 09:32:17.606053 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":60}
I20260925 09:32:17.610098 1142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.592766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50480 (local address 127.10.114.125:33051)
0925 09:32:17.592919 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.592924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.592977 (+ 53us) server_negotiation.cc:407] Connection header received
0925 09:32:17.593124 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.593127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.593174 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.593244 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.594523 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.595393 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.597115 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.597797 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.597972 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.603020 (+ 5048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:17.603027 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:17.603033 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:17.603074 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:17.606481 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.607694 (+ 1213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.607698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.607700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.607922 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.608302 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.608309 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.608312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.608817 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:17.609087 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:17.609537 (+ 450us) server_negotiation.cc:299] Negotiation successful
0925 09:32:17.609752 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 09:32:17.613034 1212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f254a73073dc4eb68845b04632a7ff8b: Bootstrap starting.
I20260925 09:32:17.613660 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.596901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41952)
0925 09:32:17.597080 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.597097 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:32:17.597188 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:32:17.598312 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:32:17.598319 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:32:17.598834 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:32:17.599050 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:17.599062 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.600872 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.600877 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:17.602371 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:17.602389 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.603331 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.603337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:17.603542 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.604168 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:32:17.604184 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:32:17.606559 (+ 2375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:32:17.610637 (+ 4078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:17.610643 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:17.610646 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:17.611021 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:17.611441 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:17.611449 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:17.611452 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:17.611578 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:17.612240 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:32:17.612245 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:32:17.612822 (+ 577us) client_negotiation.cc:769] Sending connection context
0925 09:32:17.613011 (+ 189us) client_negotiation.cc:240] Negotiation successful
0925 09:32:17.613279 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 09:32:17.613713 1218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.597245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41952 (local address 127.10.114.124:40347)
0925 09:32:17.597991 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.597996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.598016 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:32:17.598058 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.598064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.598130 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.598232 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.599237 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.600705 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.602595 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.603191 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.603320 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.606802 (+ 3482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:17.606808 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:17.606811 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:17.606842 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:17.610498 (+ 3656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.611147 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.611152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.611155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.611312 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.611689 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.611695 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.611698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.612099 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:17.612262 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:17.613099 (+ 837us) server_negotiation.cc:299] Negotiation successful
0925 09:32:17.613307 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":361,"threads_started":1}
I20260925 09:32:17.615495 10697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:32:17.616938 1212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f254a73073dc4eb68845b04632a7ff8b: Neither blocks nor log segments found. Creating new log.
I20260925 09:32:17.617969 1212 log.cc:824] T 00000000000000000000000000000000 P f254a73073dc4eb68845b04632a7ff8b: Log is configured to *not* fsync() on all Append() calls
I20260925 09:32:17.619022 1063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61: Bootstrap starting.
I20260925 09:32:17.622473 1063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61: Neither blocks nor log segments found. Creating new log.
I20260925 09:32:17.622736 1212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f254a73073dc4eb68845b04632a7ff8b: No bootstrap required, opened a new log
I20260925 09:32:17.618765 1038 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ac0833be9eaa4df4b0efd85baf930c61"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38492
I20260925 09:32:17.623605 1063 log.cc:824] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61: Log is configured to *not* fsync() on all Append() calls
I20260925 09:32:17.626848 1063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61: No bootstrap required, opened a new log
I20260925 09:32:17.630415 1212 raft_consensus.cc:348] T 00000000000000000000000000000000 P f254a73073dc4eb68845b04632a7ff8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0833be9eaa4df4b0efd85baf930c61" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } } peers { permanent_uuid: "d3c6975853704da6bd1bead42ffe6078" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } } peers { permanent_uuid: "f254a73073dc4eb68845b04632a7ff8b" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } }
I20260925 09:32:17.630892 1212 raft_consensus.cc:729] T 00000000000000000000000000000000 P f254a73073dc4eb68845b04632a7ff8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f254a73073dc4eb68845b04632a7ff8b, State: Initialized, Role: FOLLOWER
I20260925 09:32:17.631405 1212 consensus_queue.cc:260] T 00000000000000000000000000000000 P f254a73073dc4eb68845b04632a7ff8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0833be9eaa4df4b0efd85baf930c61" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } } peers { permanent_uuid: "d3c6975853704da6bd1bead42ffe6078" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } } peers { permanent_uuid: "f254a73073dc4eb68845b04632a7ff8b" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } }
I20260925 09:32:17.632494 1221 sys_catalog.cc:455] T 00000000000000000000000000000000 P f254a73073dc4eb68845b04632a7ff8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0833be9eaa4df4b0efd85baf930c61" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } } peers { permanent_uuid: "d3c6975853704da6bd1bead42ffe6078" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } } peers { permanent_uuid: "f254a73073dc4eb68845b04632a7ff8b" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } } }
I20260925 09:32:17.633035 1221 sys_catalog.cc:458] T 00000000000000000000000000000000 P f254a73073dc4eb68845b04632a7ff8b [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:32:17.633725 1212 sys_catalog.cc:564] T 00000000000000000000000000000000 P f254a73073dc4eb68845b04632a7ff8b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:32:17.634300 1063 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0833be9eaa4df4b0efd85baf930c61" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } } peers { permanent_uuid: "d3c6975853704da6bd1bead42ffe6078" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } } peers { permanent_uuid: "f254a73073dc4eb68845b04632a7ff8b" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } }
I20260925 09:32:17.634675 1063 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac0833be9eaa4df4b0efd85baf930c61, State: Initialized, Role: FOLLOWER
I20260925 09:32:17.635133 1063 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0833be9eaa4df4b0efd85baf930c61" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } } peers { permanent_uuid: "d3c6975853704da6bd1bead42ffe6078" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } } peers { permanent_uuid: "f254a73073dc4eb68845b04632a7ff8b" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } }
I20260925 09:32:17.636273 1223 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0833be9eaa4df4b0efd85baf930c61" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } } peers { permanent_uuid: "d3c6975853704da6bd1bead42ffe6078" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } } peers { permanent_uuid: "f254a73073dc4eb68845b04632a7ff8b" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } } }
I20260925 09:32:17.636829 1223 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:32:17.637300 1063 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:32:17.637399 1038 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 09:32:17.637609 1038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:32:17.641836 1038 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0833be9eaa4df4b0efd85baf930c61" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } } peers { permanent_uuid: "d3c6975853704da6bd1bead42ffe6078" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } } peers { permanent_uuid: "f254a73073dc4eb68845b04632a7ff8b" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } }
I20260925 09:32:17.643694 1038 leader_election.cc:288] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [CANDIDATE]: Term 1 election: Requested vote from peers d3c6975853704da6bd1bead42ffe6078 (127.10.114.125:33051), f254a73073dc4eb68845b04632a7ff8b (127.10.114.124:40347)
I20260925 09:32:17.645241 1187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac0833be9eaa4df4b0efd85baf930c61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f254a73073dc4eb68845b04632a7ff8b"
I20260925 09:32:17.645818 1187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f254a73073dc4eb68845b04632a7ff8b [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:32:17.650388 1187 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f254a73073dc4eb68845b04632a7ff8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac0833be9eaa4df4b0efd85baf930c61 in term 1.
I20260925 09:32:17.651187 1013 leader_election.cc:302] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac0833be9eaa4df4b0efd85baf930c61, f254a73073dc4eb68845b04632a7ff8b; no voters:
I20260925 09:32:17.651839 1223 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:32:17.655745 1223 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [term 1 LEADER]: Becoming Leader. State: Replica: ac0833be9eaa4df4b0efd85baf930c61, State: Running, Role: LEADER
I20260925 09:32:17.656448 1223 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0833be9eaa4df4b0efd85baf930c61" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } } peers { permanent_uuid: "d3c6975853704da6bd1bead42ffe6078" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } } peers { permanent_uuid: "f254a73073dc4eb68845b04632a7ff8b" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } }
I20260925 09:32:17.683791 1232 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac0833be9eaa4df4b0efd85baf930c61. Latest consensus state: current_term: 1 leader_uuid: "ac0833be9eaa4df4b0efd85baf930c61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0833be9eaa4df4b0efd85baf930c61" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } } peers { permanent_uuid: "d3c6975853704da6bd1bead42ffe6078" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } } peers { permanent_uuid: "f254a73073dc4eb68845b04632a7ff8b" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } } }
I20260925 09:32:17.684381 1232 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [sys.catalog]: This master's current role is: LEADER
I20260925 09:32:17.695250 1246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260925 09:32:17.696410 1017 tablet.cc:2369] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:32:17.701865 1249 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 09:32:17.708309 1246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:32:17.713964 1137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } attempt: 5
I20260925 09:32:17.705011 1145 master_runner.cc:186] Error getting master registration for 127.10.114.126:42915: OK, instance_id { permanent_uuid: "ac0833be9eaa4df4b0efd85baf930c61" instance_seqno: 1790328736457327 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 09:32:17.718153 1145 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 25 09:32:17 dist-test-slave-9pj8 krb5kdc[983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328736, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.114.125@KRBTEST.COM for kudu/127.10.114.124@KRBTEST.COM
I20260925 09:32:17.726784 1142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.671334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50490 (local address 127.10.114.125:33051)
0925 09:32:17.671613 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.671622 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.673938 (+ 2316us) server_negotiation.cc:407] Connection header received
0925 09:32:17.674576 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.674585 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.674676 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.674758 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.679861 (+ 5103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.681207 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.701548 (+ 20341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.702272 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.702379 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.712255 (+ 9876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:17.712262 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:17.712268 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:17.712319 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:17.715933 (+ 3614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.720955 (+ 5022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.720963 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.720968 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.721169 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.722285 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.722290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.722293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.722696 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:17.722931 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:17.726183 (+ 3252us) server_negotiation.cc:299] Negotiation successful
0925 09:32:17.726407 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260925 09:32:17.729172 1187 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f254a73073dc4eb68845b04632a7ff8b [term 1 FOLLOWER]: Refusing update from remote peer ac0833be9eaa4df4b0efd85baf930c61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:32:17.730283 1223 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f254a73073dc4eb68845b04632a7ff8b" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:32:17.731602 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.673245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38500 (local address 127.10.114.126:42915)
0925 09:32:17.673403 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.673410 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.687665 (+ 14255us) server_negotiation.cc:407] Connection header received
0925 09:32:17.688090 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.688100 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.688209 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.688302 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.692814 (+ 4512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.694295 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.703822 (+ 9527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.704526 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.704650 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.716835 (+ 12185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:17.716841 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:17.716847 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:17.716893 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:17.720897 (+ 4004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.723389 (+ 2492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.723394 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.723397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.723564 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.725130 (+ 1566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.725135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.725141 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.725486 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:17.725744 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:17.730891 (+ 5147us) server_negotiation.cc:299] Negotiation successful
0925 09:32:17.731190 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 09:32:17.732752 1215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.715242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41972 (local address 127.10.114.124:40347)
0925 09:32:17.715420 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.715426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.715443 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:32:17.715490 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.715495 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.715568 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.715663 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.716699 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.717777 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.719623 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.720341 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.720484 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.724304 (+ 3820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:17.724309 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:17.724313 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:17.724346 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:17.727507 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.728648 (+ 1141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.728654 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.728658 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.728906 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.729547 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.729553 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.729556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.730179 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:17.730449 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:17.731355 (+ 906us) server_negotiation.cc:299] Negotiation successful
0925 09:32:17.731710 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 09:32:17.733795 1141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.714513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.114.124:40347 (local address 127.0.0.1:41972)
0925 09:32:17.714889 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0925 09:32:17.714908 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:32:17.715012 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:32:17.715865 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:32:17.715871 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:32:17.716344 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:32:17.716526 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:17.716538 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.717969 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.717984 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:17.719454 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:32:17.719470 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.720675 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.720682 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:32:17.720897 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.721461 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:32:17.721488 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:32:17.724109 (+ 2621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:32:17.727639 (+ 3530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:17.727644 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:17.727647 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:17.728463 (+ 816us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:17.729120 (+ 657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:32:17.729155 (+ 35us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:32:17.729158 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:32:17.729376 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:32:17.730570 (+ 1194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:32:17.730575 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:32:17.730866 (+ 291us) client_negotiation.cc:769] Sending connection context
0925 09:32:17.731900 (+ 1034us) client_negotiation.cc:240] Negotiation successful
0925 09:32:17.732828 (+ 928us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239}
I20260925 09:32:17.740285 1137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3c6975853704da6bd1bead42ffe6078: Bootstrap starting.
I20260925 09:32:17.742484 1218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:17.674783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41966 (local address 127.10.114.124:40347)
0925 09:32:17.675043 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.675052 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.689943 (+ 14891us) server_negotiation.cc:407] Connection header received
0925 09:32:17.696761 (+ 6818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.696771 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.696912 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.697005 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.708674 (+ 11669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.709797 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.719079 (+ 9282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:17.719864 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:17.719965 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:17.735332 (+ 15367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:32:17.735338 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:32:17.735343 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:32:17.735379 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:32:17.739085 (+ 3706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.740059 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.740066 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.740071 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.740211 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:32:17.740767 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:32:17.740776 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:32:17.740780 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:32:17.741155 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:32:17.741373 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:17.741913 (+ 540us) server_negotiation.cc:299] Negotiation successful
0925 09:32:17.742140 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 09:32:17.746026 1137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3c6975853704da6bd1bead42ffe6078: Neither blocks nor log segments found. Creating new log.
I20260925 09:32:17.747364 1137 log.cc:824] T 00000000000000000000000000000000 P d3c6975853704da6bd1bead42ffe6078: Log is configured to *not* fsync() on all Append() calls
I20260925 09:32:17.749357 1221 sys_catalog.cc:455] T 00000000000000000000000000000000 P f254a73073dc4eb68845b04632a7ff8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac0833be9eaa4df4b0efd85baf930c61. Latest consensus state: current_term: 1 leader_uuid: "ac0833be9eaa4df4b0efd85baf930c61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0833be9eaa4df4b0efd85baf930c61" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } } peers { permanent_uuid: "d3c6975853704da6bd1bead42ffe6078" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } } peers { permanent_uuid: "f254a73073dc4eb68845b04632a7ff8b" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } } }
I20260925 09:32:17.749647 1221 sys_catalog.cc:458] T 00000000000000000000000000000000 P f254a73073dc4eb68845b04632a7ff8b [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:32:17.754803 1221 sys_catalog.cc:455] T 00000000000000000000000000000000 P f254a73073dc4eb68845b04632a7ff8b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ac0833be9eaa4df4b0efd85baf930c61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0833be9eaa4df4b0efd85baf930c61" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } } peers { permanent_uuid: "d3c6975853704da6bd1bead42ffe6078" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } } peers { permanent_uuid: "f254a73073dc4eb68845b04632a7ff8b" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } } }
I20260925 09:32:17.755026 1221 sys_catalog.cc:458] T 00000000000000000000000000000000 P f254a73073dc4eb68845b04632a7ff8b [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:32:17.748703 996 master_runner.cc:186] Error getting master registration for 127.10.114.125:33051: OK, instance_id { permanent_uuid: "d3c6975853704da6bd1bead42ffe6078" instance_seqno: 1790328736984803 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 09:32:17.759639 996 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 09:32:17.762426 1137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3c6975853704da6bd1bead42ffe6078: No bootstrap required, opened a new log
I20260925 09:32:17.764325 1223 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ac0833be9eaa4df4b0efd85baf930c61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0833be9eaa4df4b0efd85baf930c61" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } } peers { permanent_uuid: "d3c6975853704da6bd1bead42ffe6078" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } } peers { permanent_uuid: "f254a73073dc4eb68845b04632a7ff8b" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } } }
I20260925 09:32:17.764605 1223 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [sys.catalog]: This master's current role is: LEADER
I20260925 09:32:17.767179 1255 mvcc.cc:205] Tried to move back new op lower bound from 7333186509723729920 to 7333186509447897088. Current Snapshot: MvccSnapshot[applied={T|T < 7333186509723729920 or (T in {7333186509723729920})}]
I20260925 09:32:17.769225 1246 catalog_manager.cc:1347] Generated new cluster ID: 92f6005ac9a54a53b0bbb3f4fb1f7ee2
I20260925 09:32:17.769315 1246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:32:17.773166 1137 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3c6975853704da6bd1bead42ffe6078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0833be9eaa4df4b0efd85baf930c61" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } } peers { permanent_uuid: "d3c6975853704da6bd1bead42ffe6078" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } } peers { permanent_uuid: "f254a73073dc4eb68845b04632a7ff8b" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } }
I20260925 09:32:17.773562 1137 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3c6975853704da6bd1bead42ffe6078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c6975853704da6bd1bead42ffe6078, State: Initialized, Role: FOLLOWER
I20260925 09:32:17.774057 1137 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3c6975853704da6bd1bead42ffe6078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0833be9eaa4df4b0efd85baf930c61" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } } peers { permanent_uuid: "d3c6975853704da6bd1bead42ffe6078" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } } peers { permanent_uuid: "f254a73073dc4eb68845b04632a7ff8b" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } }
I20260925 09:32:17.775224 1263 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3c6975853704da6bd1bead42ffe6078 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0833be9eaa4df4b0efd85baf930c61" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } } peers { permanent_uuid: "d3c6975853704da6bd1bead42ffe6078" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } } peers { permanent_uuid: "f254a73073dc4eb68845b04632a7ff8b" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } } }
I20260925 09:32:17.775895 1263 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3c6975853704da6bd1bead42ffe6078 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:32:17.776292 1137 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3c6975853704da6bd1bead42ffe6078 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:32:17.776616 1111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3c6975853704da6bd1bead42ffe6078 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:32:17.780102 1111 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d3c6975853704da6bd1bead42ffe6078 [term 1 FOLLOWER]: Refusing update from remote peer ac0833be9eaa4df4b0efd85baf930c61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:32:17.781986 1262 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c6975853704da6bd1bead42ffe6078" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:32:17.783836 1112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac0833be9eaa4df4b0efd85baf930c61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d3c6975853704da6bd1bead42ffe6078"
I20260925 09:32:17.787144 1112 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d3c6975853704da6bd1bead42ffe6078 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac0833be9eaa4df4b0efd85baf930c61 in term 1.
I20260925 09:32:17.815282 1270 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3c6975853704da6bd1bead42ffe6078 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ac0833be9eaa4df4b0efd85baf930c61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0833be9eaa4df4b0efd85baf930c61" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } } peers { permanent_uuid: "d3c6975853704da6bd1bead42ffe6078" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } } peers { permanent_uuid: "f254a73073dc4eb68845b04632a7ff8b" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } } }
I20260925 09:32:17.815717 1270 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3c6975853704da6bd1bead42ffe6078 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:32:17.816134 1263 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3c6975853704da6bd1bead42ffe6078 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac0833be9eaa4df4b0efd85baf930c61. Latest consensus state: current_term: 1 leader_uuid: "ac0833be9eaa4df4b0efd85baf930c61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0833be9eaa4df4b0efd85baf930c61" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } } peers { permanent_uuid: "d3c6975853704da6bd1bead42ffe6078" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } } peers { permanent_uuid: "f254a73073dc4eb68845b04632a7ff8b" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } } }
I20260925 09:32:17.816403 1263 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3c6975853704da6bd1bead42ffe6078 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:32:17.817586 1246 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:32:17.817879 1262 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ac0833be9eaa4df4b0efd85baf930c61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0833be9eaa4df4b0efd85baf930c61" member_type: VOTER last_known_addr { host: "127.10.114.126" port: 42915 } } peers { permanent_uuid: "d3c6975853704da6bd1bead42ffe6078" member_type: VOTER last_known_addr { host: "127.10.114.125" port: 33051 } } peers { permanent_uuid: "f254a73073dc4eb68845b04632a7ff8b" member_type: VOTER last_known_addr { host: "127.10.114.124" port: 40347 } } }
I20260925 09:32:17.818229 1262 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61 [sys.catalog]: This master's current role is: LEADER
I20260925 09:32:17.819427 1246 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:32:17.833369 1246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac0833be9eaa4df4b0efd85baf930c61: Generated new TSK 0
I20260925 09:32:17.834383 1246 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:32:17.850638 1278 catalog_manager.cc:1259] Loaded cluster ID: 92f6005ac9a54a53b0bbb3f4fb1f7ee2
I20260925 09:32:17.850828 1278 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d3c6975853704da6bd1bead42ffe6078: loading cluster ID for follower catalog manager: success
I20260925 09:32:17.854561 1278 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d3c6975853704da6bd1bead42ffe6078: acquiring CA information for follower catalog manager: success
W20260925 09:32:17.855860 1278 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
W20260925 09:32:17.922333 1142 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:17.912748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50504 (local address 127.10.114.125:33051)
0925 09:32:17.912965 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.912972 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.920654 (+ 7682us) server_negotiation.cc:407] Connection header received
0925 09:32:17.920879 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.920884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.920960 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.921066 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.922080 (+ 1014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50504: BlockingRecv error: recv got EOF from 127.0.0.1:50504 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260925 09:32:17.924506 1216 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:17.912668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38510 (local address 127.10.114.126:42915)
0925 09:32:17.912863 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.912869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.920326 (+ 7457us) server_negotiation.cc:407] Connection header received
0925 09:32:17.920670 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.920677 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.920772 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.920851 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.924246 (+ 3395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38510: BlockingRecv error: recv got EOF from 127.0.0.1:38510 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260925 09:32:17.925119 1218 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:17.920828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41984 (local address 127.10.114.124:40347)
0925 09:32:17.920993 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.921000 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.923201 (+ 2201us) server_negotiation.cc:407] Connection header received
0925 09:32:17.923435 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.923441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.923524 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.923596 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.924862 (+ 1266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41984: BlockingRecv error: recv got EOF from 127.0.0.1:41984 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260925 09:32:17.932612 1142 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:17.930689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50512 (local address 127.10.114.125:33051)
0925 09:32:17.930842 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.930847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.930957 (+ 110us) server_negotiation.cc:407] Connection header received
0925 09:32:17.931242 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.931252 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.931391 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.931499 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.932329 (+ 830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50512: BlockingRecv error: recv got EOF from 127.0.0.1:50512 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260925 09:32:17.949546 1142 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:17.941971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50526 (local address 127.10.114.125:33051)
0925 09:32:17.942126 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.942131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.948193 (+ 6062us) server_negotiation.cc:407] Connection header received
0925 09:32:17.948361 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.948366 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.948447 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.948548 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.949318 (+ 770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50526: BlockingRecv error: recv got EOF from 127.0.0.1:50526 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260925 09:32:17.959504 1142 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:17.957464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50530 (local address 127.10.114.125:33051)
0925 09:32:17.957619 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.957624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.958115 (+ 491us) server_negotiation.cc:407] Connection header received
0925 09:32:17.958278 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.958283 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.958380 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.958497 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.959282 (+ 785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50530: BlockingRecv error: recv got EOF from 127.0.0.1:50530 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260925 09:32:17.975878 1142 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:17.973512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50532 (local address 127.10.114.125:33051)
0925 09:32:17.973702 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.973709 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.974171 (+ 462us) server_negotiation.cc:407] Connection header received
0925 09:32:17.974444 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.974450 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.974568 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.974652 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.975602 (+ 950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50532: BlockingRecv error: recv got EOF from 127.0.0.1:50532 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260925 09:32:17.989116 1142 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:17.986964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50538 (local address 127.10.114.125:33051)
0925 09:32:17.987118 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:17.987124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:17.987613 (+ 489us) server_negotiation.cc:407] Connection header received
0925 09:32:17.987831 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:17.987836 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:17.987930 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:17.988054 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:17.988851 (+ 797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50538: BlockingRecv error: recv got EOF from 127.0.0.1:50538 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260925 09:32:18.015055 1142 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:18.012902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50554 (local address 127.10.114.125:33051)
0925 09:32:18.013124 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:18.013134 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:18.013680 (+ 546us) server_negotiation.cc:407] Connection header received
0925 09:32:18.013910 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:18.013915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:18.014009 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:18.014121 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:18.014827 (+ 706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50554: BlockingRecv error: recv got EOF from 127.0.0.1:50554 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260925 09:32:18.052991 1142 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:18.050673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50564 (local address 127.10.114.125:33051)
0925 09:32:18.050901 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:18.050914 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:18.051444 (+ 530us) server_negotiation.cc:407] Connection header received
0925 09:32:18.051682 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:18.051686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:18.051757 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:18.051942 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:18.052690 (+ 748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50564: BlockingRecv error: recv got EOF from 127.0.0.1:50564 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260925 09:32:18.123472 1142 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:18.121123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50566 (local address 127.10.114.125:33051)
0925 09:32:18.121308 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:18.121314 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:18.121912 (+ 598us) server_negotiation.cc:407] Connection header received
0925 09:32:18.122116 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:18.122123 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:18.122229 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:18.122364 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:18.123268 (+ 904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50566: BlockingRecv error: recv got EOF from 127.0.0.1:50566 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260925 09:32:18.257113 1142 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:18.254925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50576 (local address 127.10.114.125:33051)
0925 09:32:18.255166 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:18.255172 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:18.255467 (+ 295us) server_negotiation.cc:407] Connection header received
0925 09:32:18.255698 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:18.255703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:18.255841 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:18.256000 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:18.256806 (+ 806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50576: BlockingRecv error: recv got EOF from 127.0.0.1:50576 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260925 09:32:18.518471 1142 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:32:18.516465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50590 (local address 127.10.114.125:33051)
0925 09:32:18.516645 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:18.516650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:18.517345 (+ 695us) server_negotiation.cc:407] Connection header received
0925 09:32:18.517517 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:18.517521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:18.517595 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:18.517699 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:32:18.518301 (+ 602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50590: BlockingRecv error: recv got EOF from 127.0.0.1:50590 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:32:18.706750 1249 catalog_manager.cc:1259] Loaded cluster ID: 92f6005ac9a54a53b0bbb3f4fb1f7ee2
I20260925 09:32:18.706903 1249 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f254a73073dc4eb68845b04632a7ff8b: loading cluster ID for follower catalog manager: success
I20260925 09:32:18.709563 1249 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f254a73073dc4eb68845b04632a7ff8b: acquiring CA information for follower catalog manager: success
I20260925 09:32:18.711467 1249 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f254a73073dc4eb68845b04632a7ff8b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 09:32:18.862294 1278 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d3c6975853704da6bd1bead42ffe6078: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 09:32:19.042023 1345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:19.034739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50596 (local address 127.10.114.125:33051)
0925 09:32:19.035985 (+ 1246us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:19.035990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:19.036009 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:32:19.036134 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:19.036141 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:19.036305 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:19.036428 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:32:19.036953 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:19.037908 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:19.039629 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:19.040130 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:19.040243 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:19.040789 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:32:19.041330 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:32:19.041469 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:19.041583 (+ 114us) server_negotiation.cc:299] Negotiation successful
0925 09:32:19.041652 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1126,"thread_start_us":1023,"threads_started":1}
I20260925 09:32:19.054304 1351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:19.048327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41990 (local address 127.10.114.124:40347)
0925 09:32:19.048930 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:19.048938 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:19.048959 (+ 21us) server_negotiation.cc:407] Connection header received
0925 09:32:19.049205 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:19.049209 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:19.049384 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:19.049553 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:32:19.050020 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:19.050738 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:19.052095 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:19.052512 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:19.052669 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:19.053153 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:32:19.053689 (+ 536us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:32:19.053849 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:19.053912 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 09:32:19.053987 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":316,"threads_started":1}
I20260925 09:32:19.067369 1345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:19.060097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50598 (local address 127.10.114.125:33051)
0925 09:32:19.060602 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:19.060610 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:19.061254 (+ 644us) server_negotiation.cc:407] Connection header received
0925 09:32:19.061450 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:19.061453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:19.061518 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:19.061643 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:32:19.062347 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:19.063225 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:19.065309 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:19.065953 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:19.066078 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:19.066602 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:32:19.066736 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:32:19.066847 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:19.066985 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 09:32:19.067064 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337}
I20260925 09:32:19.069079 1357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:19.060077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38520 (local address 127.10.114.126:42915)
0925 09:32:19.060605 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:19.060614 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:19.062245 (+ 1631us) server_negotiation.cc:407] Connection header received
0925 09:32:19.062399 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:19.062407 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:19.062654 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:19.062863 (+ 209us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:32:19.063462 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:19.064648 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:19.066677 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:19.067341 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:19.067465 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:19.067991 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:32:19.068439 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:32:19.068538 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:19.068740 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 09:32:19.068822 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":254,"threads_started":1}
I20260925 09:32:19.069669 1351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:32:19.062165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42006 (local address 127.10.114.124:40347)
0925 09:32:19.062332 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 09:32:19.062338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:32:19.063613 (+ 1275us) server_negotiation.cc:407] Connection header received
0925 09:32:19.063792 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:32:19.063798 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:32:19.063886 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:32:19.063976 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:32:19.064602 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:19.065537 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:19.067722 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:32:19.068325 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:32:19.068448 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:32:19.068915 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:32:19.069041 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:32:19.069105 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 09:32:19.069365 (+ 260us) server_negotiation.cc:299] Negotiation successful
0925 09:32:19.069437 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 09:32:19.073529 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 996
I20260925 09:32:19.074572 1058 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:32:19.182806 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 996
I20260925 09:32:19.204804 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 1070
I20260925 09:32:19.205917 1132 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:32:19.312202 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 1070
I20260925 09:32:19.332564 10697 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu pid 1145
I20260925 09:32:19.333977 1207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:32:19.438628 10697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMvTxiP/build/asan/bin/kudu with pid 1145
2026-09-25T09:32:19Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4745 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9812 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (354106 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST